Universal Approximation Theorems for Neural Networks Surveyed
A new survey on arXiv revisits universal approximation theorems for neural networks, tracing their evolution from qualitative density results to a rich quantitative theory. The paper covers classical results for single-hidden-layer networks and modern bounds linking approximation error to network size and smoothness assumptions. Emphasis is placed on depth-width trade-offs and the advantages of deeper architectures. The survey provides a mathematical explanation for neural networks' expressive power, asserting density in function classes like continuous functions on compact subsets of R^d, L^p spaces, and Sobolev spaces under mild activation conditions.
Key facts
- arXiv paper 2605.21451 surveys approximation theory for neural networks.
- Universal approximation theorems explain neural network expressive power.
- Feedforward networks are dense in continuous functions on compact subsets of R^d.
- Density also holds in L^p spaces and Sobolev spaces.
- Theory has evolved from qualitative to quantitative over four decades.
- Quantitative bounds relate approximation error to network size.
- Depth-width trade-offs are a key focus.
- Deeper architectures can achieve better approximation rates.
Entities
Institutions
- arXiv