Evaluation Scores Are Perishable Knowledge Claims

A recent paper on arXiv presents the idea that evaluation scores for language models should be seen as transient knowledge claims with restricted validity periods. The authors introduce "trust inflation," which occurs when combining various signals (such as automated metrics, LLM-as-judge ratings, human evaluations, and benchmark suites) through averaging leads to an overestimation of confidence based on the least reliable signal. They outline three characteristics for evaluation scores: formality (human evaluations provide stronger evidence than automated metrics), scope (results are limited to the tested distribution), and validity windows (results deteriorate as contamination increases and distributions change). The paper classifies weakest-link aggregation as a conservative endpoint in a parameterized operator family, supported by chain-of-thought analysis, possibilistic logic, and algebraic theory. This research is available on arXiv with the identifier 2607.26191.