Sharpness-Guided Equilibrium Sampling for Long-Tailed Learning
A novel technique known as Sharpness-Guided Equilibrium Sampling (SGS) has been developed to tackle the challenges of poor generalization in long-tailed learning, where dominant head classes overshadow training and tail classes settle into sharp loss minima. SGS modifies the sampling distribution by enhancing the likelihood of underrepresented classes while reducing the probabilities of those with significant SAM-induced loss variations, relying solely on cumulative class counts and EMA sharpness estimates derived from standard SAM updates. This strategy views sampling as an active control variable for optimizing geometry, eliminating the need for class-wise perturbations or extra backward passes. The method is detailed in a paper available on arXiv (2607.21999).
Key facts
- SGS addresses long-tailed learning generalization issues.
- Head classes dominate training exposure in long-tailed learning.
- Under-represented classes converge to sharper loss regions.
- SGS adjusts sampling distribution dynamically.
- Uses cumulative class counts and EMA sharpness estimates.
- No class-wise perturbations or additional backward passes needed.
- Paper available on arXiv with ID 2607.21999.
- Method treats sampling as an active control variable.
Entities
Institutions
- arXiv