MCP vs CLI: Agent Scaffolding Dominates AI Coding Costs
A new study from arXiv (2608.08654) challenges prevailing assumptions about AI coding agent costs, revealing that the scaffolding—the framework driving the agent—matters more than the interface through which tools are accessed. The researchers compared tool use via the Model Context Protocol (MCP) against a standard command-line interface (CLI) across seven agent scaffoldings and five language models. They executed one fixed software task—six operations on a private online git repository—and verified completion by inspecting the repository state, rather than relying on the agent's self-report. The results show that two of the seven scaffoldings ship no MCP support at all, completing every run using only the CLI, demonstrating that MCP is unnecessary for this class of work. These CLI-only scaffoldings were 5.0x to 28x cheaper than the five that supported MCP, indicating that the choice of scaffolding has a far greater impact on cost than the interface. Published estimates on this cost difference vary by more than an order of magnitude, often based on irreproducible practitioner reports. This study provides controlled, reproducible data, highlighting the need for rigorous benchmarking in AI tooling. The findings have significant implications for developers and organizations optimizing AI coding workflows, suggesting that focusing on scaffolding efficiency may yield greater cost savings than interface choices. The paper is available on arXiv under the identifier 2608.08654.
Key facts
- Study compares MCP and CLI tool use across seven agent scaffoldings and five language models.
- One fixed software task: six operations against a private online git repository.
- Completion verified by inspecting repository state, not agent self-report.
- Two scaffoldings have no MCP support and completed all runs using CLI only.
- CLI-only scaffoldings were 5.0x to 28x cheaper than MCP-supporting ones.
- Published estimates on cost difference vary by more than an order of magnitude.
- Study highlights that scaffolding dominates cost over interface.
- Paper available on arXiv: 2608.08654.
Entities
Institutions
- arXiv