CoinRAG: Fine-Grained KV Cache Reuse for Efficient Long-Context RAG

A new paper on arXiv (2608.07458) introduces CoinRAG, a method for optimizing Retrieval-Augmented Generation (RAG) in long-context scenarios. The approach addresses inefficiencies in existing chunk-level KV cache reuse, which suffers from information redundancy and noise. CoinRAG proposes a fine-grained, compositional reuse of offline-computed 'nugget' caches, metaphorically assembling small tokens like coins to build a compact, contextually relevant representation. The method employs two-stage retrieval to identify query-relevant semantic units within chunks, aiming to improve accuracy while maintaining low prefill latency. The paper is authored by researchers and is available as a cross-type announcement. The work targets the Pareto frontier of efficiency and accuracy in RAG systems, offering a novel solution for handling extensive retrieved contexts.