GeneFuse: A Multimodal Framework Integrating Genomic Language Models and Neuroimaging
A new multimodal learning framework named GeneFuse has been introduced by researchers to enhance the diagnosis of nervous system disorders by merging genomic language models (GLMs) with neuroimaging data. This framework tackles the issue of cross-modality heterogeneity by aligning genetic representations from pre-trained GLMs with imaging features. GeneFuse is composed of two primary elements: Genotype-Conditioned Feature Modulation (GCFM), which utilizes genomic embeddings to adjust image feature maps, and Uncertainty-aware Genomic Residual Fusion (U-GRF), a strategy that incorporates predictive uncertainty from imaging to facilitate integration. This innovative approach addresses the shortcomings of current imaging-genetics methods that simplify genetic data into rigid labels, neglecting the local sequence context of disease-related variants. The framework is elaborated in a newly submitted paper on arXiv (ID: 2608.08926), showcasing the promise of merging advanced AI with medical imaging and genomics for improved diagnostic accuracy.
Key facts
- GeneFuse is a multimodal learning framework proposed for integrating genomic and neuroimaging data.
- It uses pre-trained Genomic Language Models (GLMs) to extract genetic representations.
- The framework includes Genotype-Conditioned Feature Modulation (GCFM), a FiLM-inspired module.
- It also includes Uncertainty-aware Genomic Residual Fusion (U-GRF) for fusion.
- The method addresses cross-modality heterogeneity in imaging-genetics.
- Existing methods often encode genetic information as hard-coded labels, losing local sequence context.
- The paper is available on arXiv with ID 2608.08926.
- The research aims to improve diagnosis of nervous system diseases.
Entities
Institutions
- arXiv