Kinetic Theory for Zeroth-Order Newton Methods with Stein Correction

A recent study on arXiv introduces a kinetic framework for zeroth-order Newton-type optimization techniques, which derive gradients and Hessians from black-box function evaluations without relying on derivative data. The researchers uncover a bias in the basic random-direction Hessian estimator, even for quadratic functions, and propose a Gaussian-Stein correction to better estimate the Hessian of the Gaussian-smoothed objective. By linearizing the inverse Hessian, they identify two noise channels: gradient noise conditioned by the inverse Hessian and Hessian noise transmitted through an inverse-Hessian sandwich, with the latter featuring a second-difference factor of μ_H^{-4} when using a noisy oracle. A small-mass kinetic lift connects the finite-step Newton update to an underdamped phase-space model, while the overdamped spatial limit reveals a Lyapunov bound that highlights a curvature-variance trade-off. The paper is cataloged on arXiv under ID 2607.22567.