Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning

Key points

• Clients are not alike as they collect data at different rates, with different noise, and change at different speeds.1
• The main question is how much history should each one keep?1
• R\left(\summ wm^{3/2}\,σm\sqrt{γm/λm}\right)^{2/3}, \] for clients with weight $wm$, noise $σm$, arrival rate $λm$, and drift rate $γm$ on a domain of diameter $R$.1
• Experiments show the gain comes from the memory rather than from the solver: it persists across accelerated, stochastic-gradient, and one-step methods, and against a separately tuned common-memory baseline.1

Sources (1)

• [1]Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 11, 11:34 AM

  We study federated learning from continuously arriving data: each client samples from its own distribution, these distributions drift over time, and the server must track the minimizer of the clients' weighted average loss.

  Clients are not alike as they collect data at different rates, with different noise, and change at different speeds.

Extractive summary: sentences quoted from the sources.