ClawTrace: Cost-Aware Tracing for LLM Agent Skill Distillation
A recent research article presents ClawTrace, a platform designed for tracing agents that logs every call to an LLM, tool utilization, and the creation of sub-agents during an agent session. This information is organized into a TraceCard—a concise YAML summary detailing the USD cost per step, token counts, and redundancy indicators. CostCraft, developed from ClawTrace, serves as a distillation pipeline that analyzes TraceCards to create three varieties of skill patches: Preserve patches maintain successful behaviors; Prune patches eliminate unnecessary costly steps, each supported by a counterfactual rationale against a specific high-cost step; Repair patches address failures based on oracle evidence. Published on arXiv under ID 2604.23853, the paper highlights a significant gap in skill-distillation pipelines regarding the absence of per-step cost signals, which complicates the differentiation between adding a corrective step and removing an ineffective costly one. Experiments on 30 withheld SpreadsheetBench tasks validate the method's effectiveness.
Key facts
- ClawTrace records every LLM call, tool use, and sub-agent spawn during an agent session.
- Each session is compiled into a TraceCard: a compact YAML summary with per-step USD cost, token counts, and redundancy flags.
- CostCraft is a distillation pipeline built on ClawTrace that produces three types of skill patches: Preserve, Prune, and Repair.
- Preserve patches keep behaviors that led to success.
- Prune patches remove expensive steps that did not matter, backed by counterfactual arguments.
- Repair patches fix failures grounded in oracle evidence.
- The paper is published on arXiv with ID 2604.23853.
- Ablations were performed on 30 held-out SpreadsheetBench tasks.
Entities
Institutions
- arXiv