PRISM Benchmark Tests Language Models on Spatial-Temporal Video Generation

A new benchmark named PRISM has been developed by researchers to assess the capability of language models in generating animated outputs that are coherent both spatially and temporally through coding. This benchmark features 10,372 instruction-code pairs that have been calibrated by humans, making it 20 times larger than previous benchmarks for programmatic video generation. It is based on real-world knowledge visualization scenarios in both English and Chinese, covering 437 subject categories. PRISM employs a funnel-style evaluation framework with four key metrics: Code-Level Reliability, Spatial Reasoning, Prompt-Aware Dynamic Visual Complexity (PADVC), and Temporal Density (TD). An evaluation of seven leading LLMs highlighted a notable gap in execution and spatial reasoning, revealing that although models can create executable code, they frequently struggle with producing spatially accurate animations. This benchmark aims to rigorously evaluate the ability of language models to generate spatially correct animated outputs, essential for achieving geometric precision and temporal coherence in programmatic video generation beyond pixel-level diffusion models.