Bernstein Polynomials Enable Smooth Yet Efficient Neural Activation

A new paper on arXiv proposes a general smoothing framework for neural network activation functions using constructive approximation theory. The authors introduce the Bernstein Linear Unit (BerLU), which employs Bernstein polynomials to create a differentiable quadratic transition region. This design eliminates singularities at the origin while preserving a piecewise linear structure, balancing optimization stability with computational efficiency. The method addresses limitations of existing approaches: piecewise linear functions suffer from non-differentiability, and smooth functions incur high computational costs from transcendental operations. Theoretical analysis confirms strict continuity and differentiability. The work is relevant for deep learning practitioners seeking efficient yet stable activation functions.