Selective Prediction Mitigates Automation Bias in Clinical AI but Increases False Negatives

A new study from arXiv (2508.07617) investigates the impact of selective prediction on human decision-making in clinical settings. The research, involving 259 clinicians diagnosing and treating hospitalized patients, compared baseline performance without AI assistance to AI-assisted accuracy with and without selective prediction. Selective prediction, a method where unreliable model predictions are hidden from users, was found to reduce the negative effects of automation bias—the tendency for humans to overrely on AI predictions. However, the study also revealed that while selective prediction mitigated automation bias, it increased the rate of false negatives. The findings challenge the assumption that when AI abstains and informs the user, humans make decisions as they would without AI involvement. The study suggests that selective prediction can be a valuable tool in AI-assisted decision-making, but its trade-offs must be carefully considered. The research was announced as a replace-cross on arXiv, indicating a revision. The study's implications extend to the broader field of AI-human interaction, particularly in high-stakes environments like healthcare.