TRACE: A Two-Stage Curriculum for Parametric Tool Retrieval in Enterprise LLMs
A team of researchers has introduced TRACE, which stands for Tool Retrieval via Augmented Chain-of-thought and Enterprise rules. This new two-phase curriculum aims to tackle some major issues with parametric tool retrieval in large language models. Toolsense uses a unique virtual token for each API, but this approach can lead to slow decoding and loss of tool knowledge during training. TRACE improves on this by first using supervised fine-tuning to build tool knowledge with Low-Rank Adaptation. The second phase focuses on generating a reasoning trace before creating a JSON list of tool tokens, using enhanced data from ToolSense and expert queries. You can find the research on arXiv under the ID 2607.22639.
Key facts
- TRACE is a two-stage curriculum for parametric tool retrieval in enterprise LLMs.
- Parametric retrieval assigns each API a unique virtual token and uses constrained beam search.
- Toolsense has two drawbacks: destroys parametric tool knowledge and slow decoding.
- Stage 1 reuses multi-format memorization SFT from ToolSense with LoRA.
- Stage 2 trains the model to emit a thinking trace before producing a JSON list of tool tokens.
- Data sources include RRB pairs from ToolSense and business rule queries from domain experts.
- Training objective preserves Stage 1 MCQ and QA probing performance.
- Paper published on arXiv with identifier 2607.22639.
Entities
Institutions
- arXiv