GazeWorld: AI Model Uses Radiologist Eye-Tracking for Medical Imaging
A team of researchers has introduced GazeWorld, a novel medical imaging model that interprets radiologists' eye-tracking information as paths within an image. This model predicts latent representations of areas where the gaze is focused in an autoregressive manner and incorporates a spatial-completion component for areas that were not examined. During inference, it produces representations of patches independently of actual gaze data. The static features of GazeWorld have demonstrated exceptional diagnostic precision across nine supervised scenarios using the CheXpert, RSNA Pneumonia, and SIIM-ACR Pneumothorax datasets, achieving the highest accuracy in zero-shot evaluations.
Key facts
- GazeWorld is a medical imaging world model.
- It uses radiologist eye-tracking data as trajectories.
- It autoregressively predicts latent representations of fixated patches.
- It has a spatial-completion branch for unvisited regions.
- At inference, it generates patch representations without real gaze data.
- Frozen GazeWorld features achieve state-of-the-art diagnostic accuracy.
- Tested on CheXpert, RSNA Pneumonia, and SIIM-ACR Pneumothorax datasets.
- Achieves highest zero-shot accuracy.
Entities
Institutions
- arXiv