Inconsistency-Aware Minimization Improves Deep Learning Generalization

A new paper on arXiv introduces Inconsistency-Aware Minimization (IAM), a method to improve generalization in deep learning by leveraging unlabeled data. The authors propose a novel measure called local inconsistency, derived from an information-geometric perspective on neural network parameter spaces. Local inconsistency correlates with the generalization gap and can be computed without labels. IAM incorporates this measure into the training objective. The paper provides theoretical connections to the Fisher information matrix and loss Hessian, and empirical results show improved generalization on standard benchmarks.