SCOUT: Context-Aware Multimodal Transformer for Pathology Report Generation

SCOUT is a multimodal framework designed for generating pathology reports, taking into account context and grounded concepts. It tackles the complexities of computational pathology arising from whole-slide images (WSIs), which exhibit high resolution and diverse scales. While existing pathology foundation models can create coherent reports, they often lack clinical relevance, failing to accurately depict essential diagnostic concepts and their interrelations. By progressively conditioning image representations with global slide data and specific diagnostic concepts, SCOUT seeks to enhance interpretability and clinical relevance. The framework combines various visual evidence, ranging from detailed cellular patterns to overall tissue architecture and higher-level diagnostic ideas, aiming to deliver reports that more accurately represent pathologists' insights.