DrawVideo: Sketch-Guided Long Video Generation from Storyboards
DrawVideo is an innovative tool designed for transforming storyboard sketches into extended videos. It divides content into individual shots, each characterized by a monochrome sketch, an appearance specification, and a motion directive. The sketches dictate poses and layouts, while the appearance prompt details character aesthetics, and the motion prompt manages timing and transitions. This technique utilizes a dual-layer structure to establish reference frames, derive multiple action keyframes, and seamlessly connect adjacent clips. By addressing the limitations of traditional text-to-video systems, DrawVideo significantly improves user control over visual elements in longer video productions. The research paper is available on arXiv.
Key facts
- DrawVideo is a sketch-guided, storyboard-driven framework for controllable long-video generation.
- It decomposes long videos into independently controllable shots.
- Each shot is defined by a black-and-white sketch, an appearance prompt, and a motion prompt.
- The sketch controls pose and layout.
- The appearance prompt defines identity, scene, and style.
- The motion prompt guides temporal dynamics.
- It uses a hierarchical 'global multi-shot, local single-sketch' strategy.
- The paper is on arXiv with ID 2605.23508.
Entities
Institutions
- arXiv