Advisor Models Boost Black-Box LLM Performance by 27%
A new method called Advisor Models has been developed by researchers to train compact open-weight models that provide adaptive natural language guidance, enhancing black-box frontier language models. This technique yields a 27.4% boost in performance for GPT-5.2 on the RuleArena (Taxes) task, decreases the number of steps required by Gemini 3 Pro in SWE agent tasks by 24.6%, and surpasses static prompt optimizers in tailoring GPT-5 to individual user preferences (85-100% compared to 40-60%). The Advisor Models are both transferable and resilient, maintaining performance across various benchmarks. This approach allows for parametric optimization of black-box models without the need to alter their weights.
Key facts
- Advisor Models train small open-weight models to generate dynamic advice for black-box LLMs.
- Improves GPT-5.2's RuleArena (Taxes) performance by 27.4%.
- Reduces Gemini 3 Pro's steps in SWE agent tasks by 24.6%.
- Outperforms static prompt optimizers in personalizing GPT-5 (85-100% vs. 40-60%).
- Advisors are transferable across different student and frontier models.
- No degradation observed on other benchmarks.
- Method does not require modifying model weights.
- Published on arXiv as 2510.02453v3.
Entities
Institutions
- arXiv