ReXSonoVQA: Benchmarking VLMs on Ultrasound Video Understanding

Researchers have launched a new benchmark called ReXSonoVQA aimed at assessing vision-language models (VLMs) in ultrasound applications. This data set consists of 514 video clips, with corresponding questions—249 in a multiple-choice format and 265 as open-ended queries. The benchmark focuses on three significant aspects: understanding action goals, resolving artifacts, and contextual planning of procedures. Evaluations conducted on various models, including Gemini 3 Pro, Qwen 3.5-397B, LLaVA-Video-72B, and Seed 2.0 Pro, highlighted that while these models acquired some procedural understanding, they faced challenges in troubleshooting scenarios, showing minimal enhancements over traditional text-based assessments.