HalluAudio Benchmark Introduced to Detect Hallucinations in Audio-Language Models

Researchers have developed HalluAudio, the first comprehensive benchmark designed to evaluate hallucinations in Large Audio-Language Models (LALMs). This new tool addresses a significant gap in current assessment methods, which have primarily focused on text or vision domains while offering limited analysis for audio. HalluAudio contains over 5,000 human-verified question-answer pairs covering speech, environmental sounds, and music. The benchmark employs diverse evaluation formats including binary judgments, multi-choice reasoning, attribute verification, and open-ended questions. To systematically trigger hallucinations, the methodology incorporates adversarial prompts and mixed-audio conditions. Beyond simple accuracy metrics, the evaluation protocol measures hallucination rates and yes/no biases. This development comes as LALMs demonstrate strong performance across various audio-centric tasks, yet their tendency to generate semantically incorrect or acoustically unsupported responses remains insufficiently studied. The benchmark's creation responds to the limitations of existing audio-oriented studies, which have been constrained by small scale, narrow modality coverage, and shallow diagnostic depth.