KGCache: In-Memory Cache for Efficient KG Reasoning in LLMs

A new research paper introduces KGCache, an in-memory cache designed to optimize knowledge graph (KG) retrieval for large language models (LLMs) in question-answering tasks. The system addresses the inefficiency of repeated queries to the same graph neighborhoods, a common issue in KGQA frameworks like Think-on-Graph (ToG) and Reasoning-on-Graph (RoG). KGCache sits between the KGQA engine and the backend KG server, caching one-hop neighborhoods to serve repeated entity requests without issuing new KG queries. The paper evaluates KGCache on two benchmark datasets, WebQSP and CWQ, using cache replacement policies including LRU, LFU, and a trace-aware Oracle policy. Analysis reveals substantial entity reuse among starting entities and those reached during traversal, suggesting that caching can significantly reduce query load. The work is available on arXiv under the identifier 2608.07954.