BrainBench: New Benchmark for Comprehensive EEG Understanding in LLMs

Researchers have introduced BrainBench, a unified benchmark designed to evaluate large language models (LLMs) on comprehensive EEG understanding. The benchmark addresses the gap in current evaluations, which typically focus on isolated decoding tasks or system-specific demonstrations. BrainBench comprises four subsets: Foundational Analysis, Sleep Assessment, Neurocognitive Assessment, and Physiological Integration. It covers 17 datasets, 45 tasks, and over 100,000 real-data instances. The benchmark requires systems to perform analysis based on natural-language instructions and EEG recordings, optionally with physiological signals, and produce scientific interpretations. This initiative aims to quantify the competence of LLMs in handling complex EEG workflows that combine instruction following, signal processing, and scientific reasoning. The paper is available on arXiv under the identifier 2608.04156.