SIVR Framework Uses Sequential Internal Variance to Detect LLM Hallucinations

A new supervised framework called Sequential Internal Variance Representation (SIVR) has been developed to detect hallucinations in large language models. Unlike previous methods that rely on strict assumptions about hidden state evolution across layers, SIVR operates on a more fundamental principle: uncertainty manifests as dispersion or variance in internal representations across layers. This approach makes the method model-agnostic and task-agnostic. SIVR leverages token-wise, layer-wise features derived from hidden states, addressing limitations of techniques that focus solely on last or mean tokens and suffer from information loss. By aggregating the full sequence of per-token variance features, the framework learns temporal patterns indicative of factual errors. The method was detailed in a paper posted to arXiv with the identifier 2604.15741v1, announced as a cross submission. Uncertainty estimation through model internal states is a promising avenue for identifying unreliable LLM outputs. The research addresses critical shortcomings in existing approaches that depend on specific assumptions about how hidden states should behave.