A-PACK: Deferred Audio Pruning for Efficient Omni-Modal LLMs
A recent study presents A-PACK, a two-phase approach designed to compress multimodal sequences within omni-modal large language models (LLMs) that handle audio, video, and text simultaneously. This technique mitigates the significant computational demands associated with lengthy sequences by postponing audio pruning until after the emergence of query-conditioned multimodal interactions and utilizing local audio-visual dynamics to facilitate video compression. The framework retains audio tokens while compressing video prior to LLM processing, subsequently pruning low-relevance audio and visual tokens along with their KV-cache entries within the LLM. Tests conducted on Qwen2.5-Omni-7B and Qwen2.5-Omni-3B across four benchmarks indicate that A-PACK delivers the best average performance. The paper can be found on arXiv under the identifier 2608.08794.
Key facts
- A-PACK is a two-stage framework for omni-modal LLMs.
- It defers audio pruning until query-conditioned multimodal interactions emerge.
- Audio has higher task-relevant information density per token than video.
- Local audio-visual dynamics are more effective for visual selection than token-wise matching.
- Video is compressed before the LLM, while audio is preserved.
- Progressive pruning of low-relevance audio and visual tokens and KV-cache entries occurs inside the LLM.
- Tested on Qwen2.5-Omni-7B and Qwen2.5-Omni-3B.
- Achieves the strongest average performance across four benchmarks.
- Paper identifier: arXiv:2608.08794.
Entities
Institutions
- arXiv