Decentralized Optimization for Streaming Data via Temporal Weighting

A recent study published on arXiv introduces a decentralized gradient descent (DGD) technique designed for optimizing time-varying problems with streaming data. This method employs a temporally weighted objective that compiles all samples collected from a network of distributed agents. At each time step, every agent acquires a new sample, and the goal of the network is to follow the minimizer of this weighted objective while adhering to constraints on communication and computational resources. The authors evaluate tracking errors for losses that are both strongly convex and smooth, particularly in situations where only a limited number of DGD iterations can occur before the objective shifts. This research tackles the complexities of dynamic environments in contemporary learning systems, advancing beyond traditional fixed-objective optimization.