Scaling Laws Show Equivariance Matters More at Larger Scales

A study on neural force fields reveals that equivariant architectures, which leverage symmetry, scale better than non-equivariant models. The research shows power-law scaling behavior with architecture-dependent exponents, and higher-order representations yield better scaling. For compute-optimal training, data and model sizes should scale together regardless of architecture. The findings challenge the belief that models should discover inductive biases like symmetry on their own.