Positive Quadratic Networks: Geometry, Dynamics, and Implicit Bias

This research examines the quotient geometry associated with positive quadratic networks, characterized by a low-rank representation f_U(x) = x^T U U^T x, where U can only be identified through right orthogonal multiplication. The study links the full-column-rank stratum to the rank-r PSD manifold. For smooth objectives L(U) = ℓ(UU^T), the Euclidean factor gradient remains horizontal, allowing the factor gradient flow to align perfectly with the quotient Riemannian gradient flow. Additionally, finite-step gradient descent results in a precise congruence recursion for the predictor. In the context of quadratic regression, the effective Hessian at interpolators is shown to be the empirical measurement Gram form limited to the tangent space concerning the quotient metric. Under Gaussian rank-one measurements, the population curvature is calculated, and uniform convergence bounds are established.