Muon Optimizer Outperforms Adam in Equivariant Neural Networks
A new study from arXiv (2605.27662) compares the Muon and Adam optimizers across equivariant and geometric architectures for pointcloud and molecular learning. On ModelNet40, Muon consistently improves over Adam across all architectures. Analysis of trained checkpoints reveals that Muon yields larger Hessian curvature summaries but more regular loss surfaces, with distinct spectral properties in learned weights and intermediate representations. The research highlights the underexplored role of optimizers in equivariant networks, suggesting that optimizer choice significantly shapes learned solutions.
Key facts
- Study compares Muon and Adam optimizers on equivariant neural networks
- ModelNet40 benchmark shows Muon consistently outperforms Adam
- Analysis includes Hessian estimates, loss surface visualizations, and spectral properties
- Muon checkpoints have larger Hessian curvature but more regular loss surfaces
- Research focuses on pointcloud and molecular learning settings
- Paper available on arXiv with ID 2605.27662
- Optimizer role in equivariant networks is comparatively underexplored
- Architectural modifications like constraint relaxation are common alternative approach
Entities
Institutions
- arXiv