TransVLM: A Vision-Language Framework for Shot Transition Detection
Researchers have introduced TransVLM, a novel Vision-Language Model framework designed to detect shot transitions in video, moving beyond traditional Shot Boundary Detection (SBD) which focuses on isolated cut points. The new task, Shot Transition Detection (STD), explicitly identifies continuous temporal segments of transitions. TransVLM incorporates optical flow as a motion prior at the input stage, using a feature-fusion strategy that combines color and motion representations to enhance temporal awareness without additional visual token overhead. The work is detailed in a paper on arXiv (2604.27975).
Key facts
- TransVLM is a Vision-Language Model framework for Shot Transition Detection (STD).
- STD explicitly detects continuous temporal segments of transitions, unlike traditional SBD.
- Optical flow is injected as a motion prior at the input stage.
- A feature-fusion strategy concatenates color and motion representations.
- No additional visual token overhead is incurred on the language backbone.
- The paper is available on arXiv with ID 2604.27975.
- The approach addresses limitations of traditional SBD with complex transitions.
- TransVLM enhances temporal awareness for fine-grained inter-shot dynamics.
Entities
Institutions
- arXiv