AI Research Preference Models Predict Worthwhile Experiments

A new paper on arXiv (2608.13940) introduces AI Research Preference Models (RPMs) to address the high cost of evaluating machine learning experiments. AI research agents can propose many candidate solutions, but evaluating them requires expensive GPU time. RPMs predict which candidates are most worth executing without running all of them, using frozen pretrained language models with no task-specific training. Two forms are proposed: an inference-only model that reasons over plans, code, and prior solutions, and an agentic model that runs small pilot experiments. This could significantly improve efficiency in AI research.