Vacuity Metric Flaws in Evidential Deep Learning for OOD Detection

A new paper on arXiv (2605.06382) reveals that vacuity, or Uncertainty Mass (UM), a common metric for Out-of-Distribution (OOD) detection in Evidential Deep Learning (EDL), is fundamentally flawed due to its sensitivity to the number of classes (K). UM is calculated by dividing K by the total strength of belief (S), derived from Dirichlet parameters. The authors argue that there is no linear relationship between K and S as both increase, because EDL suppresses incorrectly assigned evidence. This causes significant discrepancies when comparing In Distribution (ID) and OOD results if K differs between them. Empirical demonstrations show AUROC and AUPR can vary substantially when class cardinality differs by just 1, with AUROC differing by as much as [value not specified in source]. The study emphasizes that K_ID and K_OOD must be equal for valid comparisons, a condition often unmet in practice.