MLLMs Outperform CNNs in Seizure Video Analysis

A recent pilot study published on arXiv investigates the effectiveness of multimodal large language models (MLLMs) in identifying pathological movements in seizure videos without prior training. Analyzing 90 clinical recordings alongside 20 semiological features defined by the ILAE, MLLMs surpassed fine-tuned CNN and ViT benchmarks on 13 out of 18 features. They showed strong performance in recognizing prominent postural and contextual elements, yet faced challenges with subtle, rapid movements. Enhancements targeting specific features, such as facial cropping, pose estimation, and audio denoising, led to improved results on 10 of the 20 features. This research underscores the promise of MLLMs for automated analysis of seizure semiology, despite challenges in detecting intricate motions.