New Paper Shows Quadratic Depth Dependence in Deep ReLU Regression

A recently released paper on arXiv (2608.17434) examines Gaussian regression utilizing the explicit vector-valued Parhi–Nowak deep-RBV² framework, characterized by depth L, width w, a layer-sum variation budget A, and an output constraint B. This architecture comprises O(Lw²) parameters, with existing error bounds differing by a factor of depth. The authors create a local packing that reveals the intrinsic quadratic depth dependence under a specific sample-size-dependent radius condition. This packing exhibits logarithmic cardinality of approximately L²w² log w, with codewords situated in an O(lambda) L² ball and pairwise spaced by Omega(lambda). Essential techniques include a bias-corrected bounded-coefficient approximation theorem and balanced amplification, which facilitates the multiplication of a depth-D ReLU network by q using a single constant channel, ensuring each coefficient increases only by q^(1/D). When applied to vector-valued RBV² blocks, this results in a layer-sum expense of O(Dw² q^(1/D)). Gaussian Fano subsequently provides a radius-explicit lower limit, elucidating the significance of depth in this regression context.