Keyless Attention: A New Mechanism to Cut KV-Cache Overhead by 50%
A new attention mechanism called Keyless Attention has been introduced by researchers, which removes key representations by implementing a specialized value-space routing projection. This innovation leads to a Value-Only Cache, cutting KV-cache memory and access overhead by 50% relative to traditional attention methods, while also enhancing decode throughput. Tests conducted on five models across four different architectures indicate that Keyless Attention either matches or surpasses standard QKV attention in perplexity for four out of five models, and it demonstrates strong performance on downstream evaluation benchmarks.
Key facts
- Keyless Attention eliminates key representations from attention computation.
- It introduces a dedicated value-space routing projection.
- Value-Only Cache reduces KV-cache memory and access overhead by 50%.
- Improves decode throughput compared to standard attention.
- Tested across five models and four architectures.
- Matches or outperforms standard QKV attention in perplexity on four of five models.
- Competitive performance on downstream evaluation benchmarks.
- Published on arXiv with ID 2606.21848.
Entities
—