New Benchmark Evaluates Text-to-Image Models for Arithmetic Education

A novel task called equation-to-visual generation has been introduced by researchers, which challenges AI to produce educationally relevant visuals from mathematical equations while maintaining their numerical and relational integrity. Drawing insights from interviews with educators and analyses of teaching materials, they developed E2V-Bench, a benchmark that includes four types of visuals and features automatic metrics for accuracy. Evaluation results indicate that recent text-to-image models often struggle, primarily due to inaccuracies in object counts and disrupted relational structures. The research also investigates strategies for performance enhancement guided by the benchmark.