Multicalibrated LLMs Enable Unbiased Prevalence Estimation Under Covariate Shift

A new arXiv paper (2604.21549) demonstrates that multicalibration, a technique from fairness research, can correct for covariate shift when estimating category prevalence using imperfect measurement devices like LLMs. Standard methods assume stable error rates across populations, but this fails under covariate shift. The authors show multicalibration, which enforces calibration conditional on input features, provides unbiased estimates. Simulations confirm standard methods exhibit bias increasing with shift magnitude, while multicalibrated estimators maintain near-zero bias. The work connects fairness theory to a longstanding measurement problem across disciplines.