EviBack: Reinforcement Learning for Agentic RAG Systems
A novel reinforcement learning technique named EviBack enhances multi-turn search within Agentic RAG systems by tackling the issue of all-zero rollout groups that fail to offer a comparative signal. This method employs an evidence-constrained Teacher backoff to provide additional supervision while maintaining verifiable Actor rewards. By decoupling evidence evaluation from answer refinement, it ensures that reference answers do not overshadow judgments based on insufficient evidence. The fully automated APE pipeline, aided by GPT-5.5, initiates with a manually created single-prompt dual-task Teacher, which then automatically organizes and labels rollout data, conducts ablation, task decomposition, evaluation, and selection, resulting in a gated two-stage Teacher. This new Teacher outperforms manual designs, enhancing downstream F1 and valid-answer rates while minimizing search efforts, duplicate queries, and forced terminations.
Key facts
- EviBack is a reinforcement learning method for Agentic RAG systems.
- It addresses all-zero rollout groups that lack comparative signal.
- Uses evidence-constrained Teacher backoff for auxiliary supervision.
- Separates evidence assessment from answer refinement.
- Prevents reference answers from overriding evidence-insufficiency judgments.
- Includes a fully automated GPT-5.5-assisted APE pipeline.
- Pipeline starts from a manually authored single-prompt dual-task Teacher.
- Improves F1 and valid-answer rate while reducing search and duplicate queries.
Entities
—