New AI Architecture SEAD Challenges Neural Network Generalization Limits
A research paper proposes the Spatiotemporal Evolution with Attractor Dynamics (SEAD) architecture, a neural cellular automaton derived from physical principles to address neural networks' failure to generalize. The work argues that generalization failures, such as an AI's inability to apply addition from 16-digit to 32-digit numbers, stem from violations of physical postulates rather than engineering flaws. Three fundamental constraints for any generalizing system are identified: Locality, requiring information to propagate at finite speed; Symmetry, demanding computational laws remain invariant across space and time; and Stability, necessitating convergence to discrete attractors that resist noise. The SEAD model implements local convolutional rules that iterate until convergence, inspired by physics. Experimental validation was conducted on three tasks, including parity, to demonstrate perfect length generalization through light-cone propagation. The paper, identified as arXiv:2602.01651v2, was announced as a replacement cross on the arXiv preprint server. This approach contrasts with traditional neural network design by deriving architecture from first principles rather than engineering it empirically.
Key facts
- The paper introduces the SEAD architecture, a neural cellular automaton.
- It addresses neural networks' failure to generalize, such as with addition from 16-digit to 32-digit numbers.
- Three physical constraints for generalization are identified: Locality, Symmetry, and Stability.
- The architecture is derived from these postulates rather than designed empirically.
- Experiments validated the theory on three tasks, including parity.
- The paper is arXiv:2602.01651v2, announced as a replace-cross on arXiv.
- Generalization failure is argued to be a violation of physical postulates, not an engineering problem.
- SEAD uses local convolutional rules iterated until convergence.
Entities
Institutions
- arXiv