Hybrid Ranker Outperforms Knowledge Graph for Agent Skill Retrieval
A new arXiv paper (2608.06196) compares two systems for retrieving skills from large libraries for AI agents. The hybrid ranker, combining lexical and dense-embedding retrieval, achieves 73.5% accuracy in top-five retrieval on 117 queries, while the typed knowledge graph performs significantly worse (-11.2 points) when used as intended. The graph's LLM-generated edges add no benefit over simple embedding neighbors. The study uses a corpus of 690 skills and highlights the inefficiency of loading entire libraries into context.
Key facts
- The paper compares two systems for agent skill retrieval over a corpus of 690 skills.
- The hybrid ranker combines lexical and dense-embedding retrieval for sparse, on-demand loading.
- The typed knowledge graph encodes workflow relations such as prerequisites, data flow, and ordering.
- On 117 realistic, non-echoing queries, the hybrid ranker retrieves the correct skill within the top five in 73.5% +/- 8.0 of cases.
- The graph is significantly worse (-11.2 points, p = 0.0007) when used as intended.
- The graph's LLM-generated edge layer adds nothing over neighbours obtained free from a local embedding pass.
- 73% of the queries were not served by the graph.
- The paper is available on arXiv with ID 2608.06196.
Entities
Institutions
- arXiv