CircuitReason-1k: New Benchmark for Visual-to-Symbolic Reasoning in Electrical Circuits
The CircuitReason-1k benchmark has been launched to assess long-horizon visual-to-symbolic reasoning in the context of electrical circuit analysis. It features 1,000 genuine textbook problems, each consisting of circuit diagrams, self-contained questions, typed or semantically defined answers, and reference solutions. The development process prioritizes an evidence-first approach to align questions, figures, and solutions, while a reasoning-focused taxonomy categorizes problems based on circuit type and dependency depth. Evaluation methods integrate conservative typed scoring with identity-blinded multi-model semantic consensus, ensuring that all problems are included in the denominator. This benchmark encompasses the entire reasoning process, addressing aspects such as symbol grounding, topology recovery, physical model selection, equation formulation, and the maintenance of units and conventions. The paper can be found on arXiv with the identifier 2608.09374.
Key facts
- Benchmark name: CircuitReason-1k
- Contains 1,000 authentic textbook problems
- Each problem includes circuit diagrams, questions, answers, and worked solutions
- Construction pipeline is evidence-first
- Taxonomy organizes problems by circuit type and dependency depth
- Evaluation uses conservative typed scoring and identity-blinded multi-model semantic consensus
- Paper available on arXiv:2608.09374
- Focuses on long-horizon visual-to-symbolic reasoning
Entities
Institutions
- arXiv