LEC: A Framework for Selection-Conditioned Risk Control in AI Predictions

The newly introduced LEC (Linear Expectation Constraints) framework tackles the issue of unreliable responses from foundation models by redefining selective prediction as a decision-making challenge constrained by linear expectations regarding selection and error indicators. This approach effectively manages the balance between the anticipated accepted errors and the expected accepted predictions, which aligns with the marginal error probability based on selection. By assuming exchangeability, a finite-sample sufficient condition is established that depends solely on a separate calibration set, providing statistical assurances that an accepted prediction will not exceed a specified risk level for error probability. The goal of this method is to supersede heuristic uncertainty estimators that inadequately differentiate between correct and incorrect outputs.