Generalization Guarantees of Randomized Block Coordinate Descent for Nonconvex Learning

Key points

• The existing theoretical analysis of RBCD mainly focuses on its convergence behavior from an optimization perspective, while its generalization behavior remains less understood for nonconvex problems.1
• In this paper, we establish high-probability generalization guarantees for RBCD with nonconvex objectives.1
• With polynomially decaying step sizes, we tune the iteration number to balance optimization and generalization, thereby quantifying how early stopping leads to the optimal convergence rate for the population gradients.1
• Finally, we develop dimension-independent bounds for loss functions of a specific structure, making the generalization guarantees applicable to high-dimensional problems.1

Sources (1)

• [1]Generalization Guarantees of Randomized Block Coordinate Descent for Nonconvex Learning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 05:00 PM

  Randomized block coordinate descent (RBCD) is a popular optimization algorithm for many large-scale machine learning problems.

  The existing theoretical analysis of RBCD mainly focuses on its convergence behavior from an optimization perspective, while its generalization behavior remains less understood for nonconvex problems.

Extractive summary: sentences quoted from the sources.