LLM Agents Factory: Retrieval-Based Domain-Specific Agent Construction
A recent study presents the LLM Agents Factory, a framework designed for on-demand creation of domain-specific LLM agents through retrieval methods. This system is built on a foundation of more than 20,000 pre-established agent profiles and offers two operational modes: retrieval via semantic search and distillation into a streamlined model for direct output generation. Results from tests conducted on MMLU, BIG-bench, and BIG-bench Hard indicate that constructing agents through retrieval methods outperforms non-agent benchmarks in terms of accuracy, while achieving AutoGen generation quality using a 120B backbone at a reduced inference cost. The research can be found on arXiv with the identifier 2608.09934.
Key facts
- LLM Agents Factory is a retrieval-based framework for constructing domain-specific LLM agents.
- It uses a base of over 20,000 predetermined agent profiles.
- The framework supports two modes: agent profile retrieval via semantic search and distillation into a compact model.
- Experiments were conducted on MMLU, BIG-bench, and BIG-bench Hard in a single-agent scenario.
- The retrieval-based agent construction surpasses non-agent baselines in accuracy.
- It matches AutoGen generation quality with a 120B backbone at a substantially lower inference cost.
- The paper is published on arXiv with identifier 2608.09934.
- The framework addresses computational cost and instability in on-the-fly agent design.
Entities
Institutions
- arXiv