Decodability Criterion Predicts When Hidden-State Selection Beats Majority Voting in LLMs

A recent study published on arXiv (2608.17124) presents CASE (Correctness-Axis SElection), a dynamic selection combiner designed for large language models. This approach involves training a linear gate based on the hidden state of answer tokens, enabling the selection of the highest-scoring candidate. The researchers tackle a critical issue in test-time information fusion: the integration of multiple sampled responses, which typically relies on majority voting. They demonstrate that this method falters on challenging questions due to correlated errors among sampled answers, often leading to incorrect selections. As a solution, they propose utilizing a correctness signal from hidden states, though its reliability is inconsistent. CASE introduces 'decodability,' a metric that effectively ranks correct responses above incorrect ones, predicting scenarios where hidden-state selection surpasses majority voting.