Utility-Aligned Embeddings Boost Dense Retrieval for RAG
Researchers propose Utility-Aligned Embeddings (UAE), a framework that trains a bi-encoder to mimic utility distributions from LLM perplexity reduction, improving retrieval precision without test-time LLM inference. On QASPER, UAE achieves 30.59% improvement in Recall@1, 30.16% in MAP, and 17.3% in Token F1 over BGE-Base.
Key facts
- UAE merges dense retrieval and utility-based re-ranking advantages.
- Formulates retrieval as distribution matching with Utility-Modulated InfoNCE.
- No test-time LLM inference required.
- QASPER benchmark: Recall@1 +30.59%, MAP +30.16%, Token F1 +17.3% vs BGE-Base.
- Addresses precision limitations in dense retrieval and noise in perplexity estimation.
Entities
Institutions
- arXiv