END: Early Noise Dropping for LLM Context Denoising
Researchers have introduced Early Noise Dropping (END), a method to improve Large Language Model (LLM) performance by removing irrelevant context without fine-tuning. END segments input into chunks and uses a linear prober on early LLM layers to identify and discard noisy chunks, preserving critical information. The approach addresses issues in retrieval-augmented generation, table question-answering, and in-context learning. The paper is available on arXiv under ID 2502.18915.
Key facts
- END segments input sequences into chunks.
- A linear prober on early LLM layers differentiates informative and noisy chunks.
- Noisy chunks are discarded early in the process.
- END does not require fine-tuning the LLM.
- The method targets retrieval-augmented generation, table question-answering, and in-context learning.
- LLMs can implicitly identify useful information at early layers.
- The paper is on arXiv with ID 2502.18915.
- END preserves critical information while removing noise.
Entities
Institutions
- arXiv