TraceCompiler: Turning Noisy LLM Agent Traces into Deterministic Workflows
Researchers have created a system called TraceCompiler that extracts and organizes noisy traces from language-model agents that use tools, turning them into mainly deterministic workflows. This addresses the problem where LLM agents often go over the same procedures, resulting in traces filled with repeated actions and unnecessary explorations. It identifies inter-tool dependencies when a consumer’s argument can be distinctly traced back to a previous producer. Each clear connection includes a verifiable evidence tuple, while unclear relationships are marked as suspicious and don’t enforce any sequence. Bindings fall into categories like constants, user inputs, and LLM choices. On the T1 dataset, this system achieved a precision of 0.928 and a recall of 0.943, with the paper available on arXiv under 2608.02680.
Key facts
- TraceCompiler mines clusters of noisy agent traces and compiles them into executable workflows.
- It admits inter-tool dependencies only when a consumer argument contains a value uniquely attributable to an earlier producer.
- Hard edges carry auditable evidence tuples; ambiguous relations are marked suspected and impose no ordering constraint.
- Bindings are classified as constants, user inputs, copied outputs, transforms, or residual LLM decisions.
- On T1, the rule recovers producer-consumer dependencies at 0.928 precision and 0.943 recall over 15,775 def-use edges.
- Baseline F1 scores are 0.711 for adjacency and 0.712 for frequency-based methods.
- The paper is available on arXiv with identifier 2608.02680.
- The system aims to reduce redundant exploration in LLM agents.
Entities
Institutions
- arXiv