ReMem: Training-Free Long Video Understanding via Temporal Granularity-Adaptive Keyframe Selection

A novel framework named ReMem (Reasoning with Memory) has been introduced for understanding long videos without the need for training. This framework tackles the challenges posed by Multimodal Large Language Models (MLLMs), which have limited context windows that restrict their capability to analyze lengthy videos. ReMem features a dual-level memory-augmented adaptation: at the query level, Memory-Driven Question Parsing utilizes the long-term memory of LLMs to interpret the temporal granularity of questions and identify semantic entities; at the video level, Synergistic Dual-Semantic Frame Alignment leverages intrinsic structural memory to synchronize frames with query semantics, facilitating Structure-Aware Dynamic Frame Routing for frame clustering. Designed for Long Video Question Answering (LongVideoQA), it tailors keyframe selection according to query temporal granularity, addressing the drawbacks of uniform sampling or static query-guided selection. The paper can be accessed on arXiv under ID 2607.24794.