Self-Speculating Agents: Joint Agent-Speculator RL for Faster Tool Calls
A new reinforcement learning method trains a single large language model to both solve tasks and predict its own next tool call, reducing latency by pre-executing calls when predictions match. The approach, called joint agent-speculator RL, unifies agent and speculator within the same model, reusing prefix KV cache. It addresses the speculator-agent gap where separate draft models or cached traces are poorly aligned with the deployed agent's behavior. The target agent itself is identified as a strong next-call speculator, leading to a simpler design.
Key facts
- arXiv paper 2607.25816 introduces self-speculating agents
- Joint agent-speculator reinforcement learning method unifies agent and speculator in one model
- Tool-call speculation predicts and pre-executes an agent's next tool call
- Existing speculators are separate draft models or cached traces
- Target agent itself is a strong next-call speculator
- Prefix KV cache is fully reused in the dual-mode model
- Method aims to reduce wall-clock time waiting for tool call results
- Speculator-agent gap is identified as a key problem
Entities
Institutions
- arXiv