LLM Uncertainty Alignment with Human Judgments

A new study on arXiv evaluates how well inference-time uncertainty measures in large language models align with human uncertainty. The researchers tested both established and novel metrics, finding that many measures strongly correlate with human group-level uncertainty, even when they do not match human answer preferences. The work highlights the gap between model calibration and human-aligned uncertainty, suggesting that inference-time signals could improve user trust and model control. The paper is available at arXiv:2508.08204.