ContextGuard: Token Pruning for Omni-LLMs

A team of researchers has introduced ContextGuard, a framework designed for token pruning during inference in Omni-modal Large Language Models (Omni-LLMs). These models handle vast amounts of multimodal input tokens, resulting in significant computational demands. Current token pruning techniques focus on selecting tokens that are crucial for the specific query or that correspond with cross-modal indicators, often overlooking essential evidence for other inquiries or wider contexts. ContextGuard approaches token reduction by maintaining extensive audio-visual context while eliminating redundant cross-modal information. It estimates broad visual semantics from audio and discards video tokens whose meanings can likely be inferred from the audio. This research has been published on arXiv under ID 2605.11605.