DebiasRAG: Tuning-Free Debiasing for LLMs via Retrieval-Augmented Generation
Researchers propose DebiasRAG, a novel framework that leverages retrieval-augmented generation (RAG) to reduce social biases in large language models (LLMs) without fine-tuning. LLMs often produce biased content related to race, gender, and age due to their training data. Existing debiasing methods like fine-tuning and prompt engineering require extra resources or domain knowledge and may degrade model performance. DebiasRAG dynamically retrieves query-specific, debiasing contexts to guide generation, preserving the LLM's original capabilities while improving fairness. The framework is tuning-free and adapts to different queries, addressing the need for dynamic debiasing. The paper is published on arXiv under ID 2605.16113.
Key facts
- DebiasRAG is a tuning-free debiasing framework based on retrieval-augmented generation.
- LLMs produce social biases involving race, gender, and age from training data.
- Fine-tuning and prompt engineering require additional resources and may degrade LLM capabilities.
- DebiasRAG dynamically retrieves query-specific debiasing contexts.
- The framework preserves the original capabilities of LLMs.
- The paper is available on arXiv with ID 2605.16113.
- DebiasRAG aims for fairer inference without fine-tuning.
- The approach addresses the need for dynamic debiasing contexts.
Entities
Institutions
- arXiv