Homogeneous Multi-Agent LLM Debate Fails to Filter Hallucinations

A recent study indicates that multi-agent debates among uniform large language models (LLMs) do not effectively eliminate hallucinations, which contradicts prevalent beliefs. Researchers from an unnamed institution performed controlled tests using teams of ten identical LLMs—Qwen2.5-7B, Llama-3.1-8B, and Ministral-3-8B—across three rounds of debate on two challenging benchmarks, GSM-Hard and MMLU-Hard. They assessed peer debates against isolated self-correction and a stochastic noise control that introduced rationales from unrelated issues. The research highlights three pathways of failure: sycophantic conformity (up to 85.5% modal adoption), contextual fragility (vulnerability rate reaching 70.0%), and consensus collapse, where plurality voting results in incorrect agreements. These results imply that isolated self-correction frequently surpasses unguided homogeneous multi-agent debates, questioning the effectiveness of current LLM debate methods.