Individually Calibrated Models Can Become Collectively Miscalibrated
A recent preprint on arXiv (2605.18858) demonstrates that when individually calibrated probabilistic prediction models engage in strategic interactions, they can collectively become miscalibrated. The researchers establish that, when aggregating based on Brier scores with positively correlated beliefs, the optimal report from each agent consistently underestimates the probability of the positive class. This results in a Price of Anarchy exceeding one whenever Cov(b_i, b_j) is greater than zero. In a standard scenario involving five agents with a pairwise correlation of 0.5, the miscalibration is pronounced. This issue occurs naturally when agents are trained independently on shared data sets, without any intentional coordination.
Key facts
- arXiv preprint 2605.18858
- Individually calibrated models can become collectively miscalibrated
- Strategic interaction in multi-agent settings causes miscalibration
- Brier-score-based aggregation with positively correlated beliefs leads to underestimation
- Price of Anarchy > 1 when Cov(b_i, b_j) > 0
- Canonical setting: n=5 agents, pairwise correlation=0.5
- Phenomenon arises from independent training on overlapping data
- No deliberate coordination required
Entities
Institutions
- arXiv