ToC-Guided Page Retrieval Boosts RAG Answer Quality

A recent study has unveiled a method for page retrieval in RAG systems that utilizes table-of-contents (ToC) guidance, deriving headings from visual formatting without the need for LLM calls. An analysis involving 1,280 scenarios across 8 enterprise documents (ranging from 5 to 195 pages) revealed that ToC enhances answer quality significantly (d = +0.41, p = 0.031), particularly in terms of completeness (+0.40) and usefulness (+0.40). When paired with answer-side verification, it surpasses the effectiveness of query-side decomposition alongside verification (d = +0.32, p = 0.036). ToC accounts for 20% of citations while only increasing the page count by 2.9 per query. Notably, improvements are more pronounced in longer documents (up to +1.50 on 118 pages), although this trend lacks statistical significance. The approach eliminates the need for multiple LLM calls by loading entire page sections rather than smaller chunks.