LLM Tool Accuracy Degrades Before Advertised Context Limits

A recent investigation conducted by Paulsen reveals that the accuracy of large language models (LLMs) in developer tools declines significantly before they reach their claimed context window sizes. The study introduces the Maximum Effective Context Window (MECW), which is notably smaller than the advertised limits. Contemporary software repositories often include substantial non-code elements—such as compiled datasets, binary model weights, minified JavaScript bundles, and extensive log files—that can exceed the context window, pushing aside pertinent source code. To tackle this issue, a framework focused on correctness-aware context hygiene is suggested: a heuristic filter based on size that operates prior to tokenization during repository scans. This filter relies solely on OS-level stat() metadata with minimal overhead, eliminating the need for index creation or query-time inference used in semantic retrieval methods like RepoCoder, GraphRAG, and AST-based chunking. The goal of this framework is to enhance the quality of context construction by filtering out irrelevant large files, thereby maintaining the effective context for relevant code.