Triple-Robustness Analysis of RAG for Multi-Hop Traceability

A new paper on arXiv (2608.05153) dives deep into retrieval-augmented generation (RAG) systems, focusing on how traceability works across multiple steps. The study sticks to a fixed retrieval setup but tweaks three different areas: the embedder, which varies from local e5-small to Azure text-embedding-3-small; the corpus, shifting from DO-178C requirements to Wikipedia paragraph chains via MuSiQue; and the judge, where they compare GPT-5.4 and GPT-4.1. They ran 4,440 main-matrix tests, 600 cross-corpus tests, and 1,200 evaluations on faithfulness. Key findings show that over-citation is a common feature, with GraphRAG producing 11-15 IDs per response but citation precision ranging from 0.12 to 0.23. Faithfulness fluctuates significantly, dropping from 74% to 40% on DO-178C but rising from 42% to 58% on Wikipedia. The paper briefly touches on stratum-con results but lacks details. Overall, it highlights how findings about GraphRAG’s citation precision issues depend on the corpus used.