On Global Joint Lower Bounds in Zeroth-Order Nonconvex Optimization

Key points

• Although the dimension dependence and the first-order dependence on $(δ,ε)$ are separately known to be unavoidable, a matching zeroth-order lower bound has remained open.1
• For smooth stochastic nonconvex optimization, we establish a scalar-oracle lower bound of $Ω\!\left(dL0^2\min\{ΔH,L0^2\}/ε^4\right)$ assuming a smooth, globally lower-bounded population objective and a second-moment bound on sample Lipschitz constants.1
• A smooth reduction then yields a lower bound for Goldstein stationarity, proving the joint minimax optimality of the dependence on $(d,δ,ε)$ under explicit high-dimensional and small-accuracy conditions.1
• Our proof combines a globally defined sequential nonconvex construction with an exact Gaussian scalar channel, multistage random-field estimates, and an antithetic finite-sample realization that preserves information hardness under adaptive reuse.1

Sources (1)

• [1]On Global Joint Lower Bounds in Zeroth-Order Nonconvex Optimization
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 06:20 AM

  Recent work has sought to transfer first-order nonconvex optimization guarantees to zeroth-order methods through gradient estimators, typically incurring a factor linear in $d$. \citet{kornowski2024} obtained an $\mathcal{O}(dδ^{-1}ε^{-3})$ oracle complexity for nonsmooth nonconvex optimization and asked whether its dependence on all three parameters is jointly optimal.

  Although the dimension dependence and the first-order dependence on $(δ,ε)$ are separately known to be unavoidable, a matching zeroth-order lower bound has remained open.

Extractive summary: sentences quoted from the sources.