Dual Hybrid Semantic Data Lake Architecture for Medical Data Harmonization with LLM-Driven Metadata Annotation

A new arXiv paper (2608.08056v1) proposes a dual hybrid semantic data lake architecture to address medical data harmonization challenges. The system integrates a knowledge graph with a human-in-the-loop verified, LLM-driven metadata annotation system. Medical data heterogeneity spans modalities (images, text, time series), diverse tabular schemata across institutions, and unstructured clinical notes. Data lakes consolidate such data without imposing a schema, but risk becoming 'data swamps' due to poor metadata. The proposed architecture leverages knowledge graphs for dynamic relationship representation and LLMs to automate metadata generation, with human verification to ensure accuracy. This approach aims to balance flexibility and integrity, offering a solution to the data swamp problem. The paper is available on arXiv under the identifier 2608.08056.