SAGE: Global Semantic Explanations for Attention-Based Survival Models in Pathology
A new framework known as Semantic Attention Global Explanations (SAGE) has been developed by researchers to deliver comprehensive, language-based explanations for attention-based multiple instance learning (ABMIL) models in computational pathology. While ABMIL is the leading method for slide-level predictions, its attention maps provide only localized insights, failing to clarify which histological features influence predictions or how the model performs across a patient population. SAGE remedies this by employing a pathology vision-language model to evaluate image patches against a set of 25 histological concepts, aggregating scores based on learned attention, and measuring the relationship of each concept to prediction risk across cohorts. This framework was tested on survival predictions from seven TCGA cancer cohorts and three foundation models, successfully identifying known prognostic indicators like the negative impact of necrosis. This advancement improves the interpretability of AI in medical imaging, essential for fostering clinical trust and adoption. The research can be found on arXiv with the identifier 2608.02803.
Key facts
- SAGE is a post-hoc framework for global explanations of ABMIL models.
- It uses a pathology vision-language model to score image patches against 25 histological concepts.
- SAGE aggregates scores according to the model's learned attention.
- It quantifies how each concept relates to prediction risk across a cohort.
- Applied to survival prediction using seven TCGA cancer cohorts.
- Three foundation models were used in the evaluation.
- SAGE recovered established prognostic features, such as the adverse association of necrosis.
- The paper is available on arXiv with ID 2608.02803.
Entities
Institutions
- arXiv
- TCGA