Mixture-of-Expert Blocks Show Strong Hallucination Detection Signals
Researchers have introduced InnerExpert, a method that uses Mixture-of-Experts (MoE) internal signals to detect hallucinations at the token level in large language models. The paper, posted on arXiv (2608.17687), argues that existing detection methods work at answer or sentence level, but per-token detection can localize hallucinated spans. In MoE architectures, a single forward pass activates sparse experts through a routing mechanism, yielding signals like router entropy, expert disagreement, and usage patterns—unavailable in dense models. InnerExpert is described as the first to exploit these signals for per-token detection. The work addresses LLMs' tendency to generate plausible but false content, a fundamental limitation.
Key facts
- The paper is available on arXiv with ID 2608.17687.
- InnerExpert is the first method to leverage MoE-specific signals for per-token hallucination detection.
- MoE architectures activate a sparse subset of experts per layer via a routing mechanism.
- Internal signals include router entropy, expert disagreement, and expert usage patterns.
- Existing detection methods operate at the answer or sentence level.
- Per-token detection is essential for localizing hallucinated spans and enabling fine-grained interventions.
- MoE-specific signals are unavailable in dense architectures.
- The research addresses LLM hallucinations, the generation of plausible but false content.
Entities
—