Group Distributionally Robust Optimization over Decentralized Networks

Key points

• In this paper, we directly optimize the original GDRO objective and establish guarantees for local models in semi-decentralized (SemiDec) and fully decentralized (Dec) settings.1
• To address this challenge, we develop a novel error decomposition that recasts decentralized GDRO as (i) a decentralized online convex optimization (D-OCO) problem and a prediction-with-expert-advice problem in SemiDec, and as (ii) two coupled D-OCO problems in Dec, with discrepancy terms capturing inconsistencies among local decisions.1
• By isolating consensus errors in analysis, we develop modular frameworks that support different consensus routines.1
• We then establish the first per-worker convergence guarantees for optimizing the original GDRO objective over decentralized networks.1

Sources (1)

• [1]Group Distributionally Robust Optimization over Decentralized Networks
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 06:13 AM

  We study group distributionally robust optimization (GDRO) over decentralized networks, where workers collaborate to learn robust models by minimizing the maximum local risk.

  In this paper, we directly optimize the original GDRO objective and establish guarantees for local models in semi-decentralized (SemiDec) and fully decentralized (Dec) settings.

Extractive summary: sentences quoted from the sources.