Matching Principle Unifies Robustness Methods in Representation Learning
A new paper on arXiv (2605.22800) presents the Matching Principle, a geometric approach that combines robustness, domain adaptation, and invariance into one statistical problem: estimating the covariance of deployment nuisance while keeping labels intact and properly regulating the encoder Jacobian. The authors show that methods like CORAL, adversarial training, IRM, data augmentation, metric learning, Jacobian penalties, and alignment constraints all serve as different ways to estimate the same idea. Within the linear-Gaussian model, they achieve closed-form optimality, including cube-root water-filling in the matched range, and highlight the importance of covering the range for quadratic Jacobian penalties. The theory also reveals a split of ranges at deep global minima.
Key facts
- Paper published on arXiv with ID 2605.22800
- Proposes the Matching Principle for nuisance-robust representation learning
- Unifies robustness, domain adaptation, photometric invariance, occlusion invariance, compositional generalization, temporal robustness, alignment safety, and anisotropic regularization
- Shows CORAL, adversarial training, IRM, augmentation, metric learning, Jacobian penalties, and alignment constraints are estimators of the same covariance object
- Proves closed-form optimality in linear-Gaussian model (Theorem A)
- Includes cube-root water-filling within the matched range
- Establishes necessity of range coverage for quadratic Jacobian penalties (Theorem G)
- Reveals range dichotomy at deep global minima
Entities
Institutions
- arXiv