RotateK: Rotation-Based Key Channel Pruning for Efficient VLM Inference

Researchers have introduced RotateK, a framework for structured Key channel pruning based on rotation, aimed at alleviating KV cache pressure during inference in Vision-Language Models (VLMs). These models transform a single image into thousands of tokens, resulting in significant memory consumption. Current token pruning techniques tend to eliminate visual information, which negatively impacts fine-grained perception tasks. By leveraging feature sparsity, RotateK compresses the channel dimension, thereby retaining more visual tokens within a fixed KV cache budget. It employs an online PCA-based rotation to synchronize token-dependent channel importance into a unified low-dimensional subspace, facilitating precise pruning with a lightweight, hardware-friendly head-wise structure. This approach balances the expressive nature of unstructured token-wise pruning with the robustness of head-wise methods. The full details can be found in arXiv:2605.19218.