Task-Adaptive Pruning: New Method for Vision Transformers

A recent paper published on arXiv (2608.10989) presents Task-Adaptive Pruning (TAP), a novel approach for token pruning in Vision Transformers. The findings indicate that pruning strategies tailored for one task do not effectively transfer across various applications such as image classification, semantic segmentation, and object detection. The use of controlled probes, which maintain the no-pruning checkpoint while applying parameter-free reduction criteria layer by layer without retraining, highlights three significant differences: segmentation and detection utilize distinct rank pruning criteria, classification is notably affected by attention-based pruning in earlier layers, and dense tasks favor contrasting recovery endpoints. TAP resolves these challenges by implementing a task register for each task, activating only the relevant one while evolving its state, enhancing existing register tokens to be task-specific. This announcement is cross-type and can be accessed via the provided URL.