Study Reveals 'Regression Tax' When Adding Skills to LLM Agents
A recent study featured on arXiv (2607.22520) analyzes the effects of incorporating procedural skills into LLM agents, uncovering a phenomenon termed 'regression tax,' where the addition of skills can actually hinder performance. Researchers conducted nearly 6,000 tests using two office automation benchmarks and three model harness stacks, comparing agents equipped with skills against those without. They differentiate between regressions—tasks that were completed without skills but failed after their introduction—and residual failures, which occur regardless of skill presence. The top-performing skills excel not by enhancing capabilities but by reducing regressions. Three identified regression factors include skill description osmosis, grounding displacement, and a third unspecified cause. The findings suggest that average improvement metrics may obscure substantial costs, advocating for a more detailed assessment of skill integration in LLM agents.
Key facts
- Study published on arXiv with ID 2607.22520
- Compares agents with and without skills across nearly 6,000 runs
- Uses two office automation benchmarks and three model harness stacks
- Defines regressions as tasks solved without skills but failed after adding skills
- Defines residual failures as tasks that fail both with and without skills
- Best-performing skills outperform by regressing less, not by gaining more
- Identifies three causes of regression: skill description osmosis, grounding displacement, and a third cause
- Skill description osmosis: a skill changes behavior simply by being present in context even when never invoked
Entities
Institutions
- arXiv