Human-Centered Benchmarking Framework for Eye-State Recognition Models
A new study has introduced the Human-Centered Benchmarking Framework (HCBF), which aims to evaluate eye-state recognition models used in driver monitoring in a more nuanced way than just overall scores. This framework highlights key factors like robustness, transferability, embedded latency, and faithfulness of explanations, rather than just looking at basic performance. The team tested six compact convolutional and transformer models using a specific protocol along with various image corruptions and training methods, evaluating them on the RT-BENE dataset. They found that clean MRL Macro-F1 scores ranged from 0.9566 to 0.9794, and zero-shot RT-BENE Macro-F1 scores fell between 0.2066 and 0.7771. The study stresses the importance of considering operational constraints in model selection for safety-critical applications. You can find the paper on arXiv with the identifier 2606.08123.
Key facts
- Introduces Human-Centered Benchmarking Framework (HCBF)
- Evaluates six compact convolutional and transformer-oriented eye-state recognition models
- Uses subject-disjoint MRL Eye protocol
- Includes deterministic image corruptions, zero-shot transfer, and participant-safe target-domain training
- Out-of-fold evaluation on RT-BENE
- TensorRT FP32 inference on NVIDIA Jetson Nano
- Black-box RISE faithfulness considered
- Clean MRL Macro-F1 ranged from 0.9566 to 0.9794
- Zero-shot RT-BENE Macro-F1 ranged from 0.2066 to 0.7771
- Matched target-domain effects varied from -0.0406 to 0.4807
Entities
Institutions
- arXiv