MM-ShiftKV: Decode-Aware KV Cache Selection for MLLMs

A new training-free method called MM-ShiftKV addresses the memory bottleneck of key-value (KV) caching in multimodal large language models (MLLMs). The method is presented in a preprint on arXiv (2607.22586v1). Standard prefill-stage KV selection assumes that queries during prefilling are representative of decoding-time queries, but this assumption fails in multimodal inference due to higher variance in decoding queries. MM-ShiftKV is a decode-aware, strictly prefill-only selection method that approximates decoding-time query distributions to improve KV importance estimation. This prevents the disproportionate discarding of semantically critical visual tokens under tight cache budgets, enhancing grounding and reasoning performance.