Verbalized Uncertainty for Risk-Controlled Deferral in Small Language Models

A recent preprint on arXiv (2608.05064) explores the role of expressed confidence in facilitating risk-controlled deferral for small open-weight language models, which are being utilized more frequently in private, offline, and budget-sensitive environments. The research assesses eleven instruction-tuned models from three different categories, with parameter sizes ranging from 0.5B to 14B, across ARC-Challenge and TruthfulQA, resulting in 25,168 local predictions. The authors present three key theoretical findings: strict monotone calibration maintains the risk-coverage frontier and error-detection AUROC; temperature scaling is ineffective for models with confidence exceeding one half while accuracy declines; and a Clopper-Pearson method transforms a 200-question calibration dataset into a finite-sample risk certificate under the i.i.d. assumption. Notably, eight out of 22 model-task combinations reached the temperature-scaling infeasibility limit within the study's parameters. The paper can be accessed on arXiv with the identifier 2608.05064.