SuperScout: Cost-Aware Routing of Coding Agents by Scouting the Repository First
SuperScout, a novel system, seeks to lower the expenses associated with utilizing frontier language models for software issues at the repository level. Rather than choosing a model based solely on the issue description, SuperScout initially examines the repository with a 7B searcher model known as SuperScout-7B, which creates a structured handoff. Claims made during this handoff are verified in a sandbox, eliminating any inaccuracies before they are sent out. The hidden states from the searcher and the task text are used by a resume-based router that assigns the task to one of four frontier fixers, with the addition of new fixers not requiring retraining. In the complete Python slice of SWE-bench Pro (266 tasks), SuperScout achieves a solve rate of 159 out of 266, comparable to the best single model's 158, while costing approximately one-fifth per solve. This configuration outperforms the random traffic-splitting baseline. The research paper can be found on arXiv under ID 2608.04804.
Key facts
- SuperScout routes coding agents after scouting the repository, not just from issue text.
- SuperScout-7B is a 7B searcher that explores the repository and produces a structured handoff.
- Handoff reproduction claims are sandbox-verified and false claims are stripped.
- A resume-based router uses searcher hidden states and task text to dispatch to one of four frontier fixers.
- Adding a new fixer requires no retraining.
- On SWE-bench Pro Python slice (266 tasks), SuperScout solves 159 tasks vs 158 for best single model.
- SuperScout achieves about a fifth of the total cost per solve compared to the best model.
- The configuration sits above the random traffic-splitting baseline.
Entities
Institutions
- arXiv