CADEngBench: New Benchmark Tests Parametric Design, Assembly Reasoning, and Physics Simulation
The introduction of CADEngBench marks a significant advancement in assessing the engineering-level quality of CAD models, extending beyond simple visual accuracy. This benchmark, outlined in arXiv paper 2608.09296, features two distinct tracks: CADEngBench-P and CADEngBench-A. CADEngBench-P evaluates 300 parametric components through 600 tasks, which involve zero-to-CAD generation and functional modifications, employing boundary-representation (B-Rep) validity, engineering checks, design-for-manufacturing (DFM) assessments, parameter-family variations, and linear-static finite element analysis (FEA) with CalculiX. Meanwhile, CADEngBench-A focuses on 150 body pairs, assessing ranked joint retrieval, precise face-and-edge grounding, joint-frame prediction, and kinematic verification. The research examined eight multimodal, code-capable models, revealing that modifying existing CAD models is significantly easier than creating them from the ground up, although intricate edits still pose difficulties. The benchmark's goal is to ensure CAD models meet design specifications, respond consistently to parameter adjustments, allow for controlled modifications, align with reference structural behaviors, and connect with other components via valid joints.
Key facts
- CADEngBench is a two-track benchmark for evaluating engineering-grade CAD models.
- CADEngBench-P evaluates 300 parametric parts with 600 tasks (zero-to-CAD and functional editing).
- CADEngBench-A evaluates 150 body pairs for assembly reasoning.
- The benchmark uses B-Rep validity, DFM checks, parameter perturbations, and FEA in CalculiX.
- Eight multimodal, code-capable models were tested.
- Editing supplied CAD is easier than generating it from scratch.
- Complex edits remain a challenge for current models.
- The paper is available on arXiv with ID 2608.09296.
Entities
Institutions
- arXiv