Non-Acceleration of the Heavy-Ball Method in 1D

Key points

• (2025)] made progress regarding acceleration in the context of smooth, strongly convex minimization: for a small enough inverse condition number, either the method does not accelerate on quadratics, or there exists a smooth strongly convex function such that heavy-ball cycles over the roots of unity.1
• The constructed functions have two variables, thus leaving the question of acceleration over univariate functions open.1
• In this work, we use a Fourier-based analysis to i) provide a proof for the non-acceleration of the heavy-ball method over the class of univariate smooth strongly convex functions.1
• We also ii) provide a strict enlargement of the previous root-of-unity cycling region, with insights regarding the "fractional" sets of Goujaud et al. [arXiv:2502.19916], and iii) give a higher dimensional cycling instance which is not covered by root-of-unity cycling.1

Sources (1)

• [1]Non-Acceleration of the Heavy-Ball Method in 1D
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 08:29 AM

  Despite having an asymptotically optimal rate of convergence on convex quadratics, the acceleration of Polyak's heavy-ball method and a characterization of its dynamics within more general settings have remained long standing open problems.

  (2025)] made progress regarding acceleration in the context of smooth, strongly convex minimization: for a small enough inverse condition number, either the method does not accelerate on quadratics, or there exists a smooth strongly convex function such that heavy-ball cycles over the roots of unity.

Extractive summary: sentences quoted from the sources.