Unified 2D Framework Integrates LLMs for Lesion Detection, Segmentation, and Report Generation

A novel 2D lesion analysis framework has been created by researchers, combining large language models (LLMs) for tasks such as reasoning, lesion bounding box detection, segmentation, and generating radiology reports. This framework was evaluated using the original DeepLesion dataset, yielding a detection mAP50 of 70.1%, a segmentation Dice score of 62.6%, and a BLEU_1 score of 64.3% for report generation. Remarkably, it surpassed the nnUNet model by 28.5% in segmentation Dice score. This research builds on prior work that utilized LLMs for lesion segmentation with the ULS23 DeepLesion dataset. The study tackles the complexities of segmentation within the original DeepLesion dataset, incorporating both spatial and anatomical context. The findings are published on arXiv with the identifier 2608.02805.