Parameter-Free Dynamic Regret for OCO Under Heavy-Tailed Noise
A new algorithm, HT-PAder, achieves universal dynamic regret in online convex optimization under heavy-tailed noise without requiring prior knowledge of problem parameters. The method combines restarted AdaGrad experts with a pathwise meta-algorithm, AdaGrad-Hedge, and handles stochastic gradients with only a finite p-th central moment for p in (1,2]. The expected regret bound scales with domain diameter D, Lipschitz constant G, noise level σ, and comparator path length P_T. This work resolves an open challenge in non-stationary OCO.
Key facts
- HT-PAder is parameter-free and requires no prior knowledge of problem parameters.
- The algorithm handles heavy-tailed noise with finite p-th central moment for p in (1,2].
- It combines restarted AdaGrad experts over geometric block lengths with AdaGrad-Hedge meta-algorithm.
- Expected universal dynamic regret is O~(GD√(T(1+P_T/D)) + σD T^{1/p}(1+P_T/D)^{(p-1)/p}).
- The paper is published on arXiv with ID 2607.27073.
- The work addresses non-stationary environments in online convex optimization.
- AdaGrad-Hedge requires no moment conditions on meta-losses.
- The algorithm achieves dynamic regret without knowing D, G, σ, or P_T.
Entities
Institutions
- arXiv