Uniform convergence at optimal rate and scale

Key points

• Uniform convergence asks how many random samples suffice for empirical averages to approximate expectations simultaneously over an entire class of functions.1
• For dimension $d$, the sharp worst-case sample complexity for additive error $\varepsilon$ is of order $d/\varepsilon^2$.1
• The challenge is to achieve error $γ+\varepsilon$ with order $d/\varepsilon^2$ samples, matching the sample bound for binary classes, where $d$ is now the fat-shattering dimension at scale $γ$.1
• Second, we derive improved covering bounds in both discrete and real-valued settings: we extend Haussler's covering lemma to partial concept classes and use this extension to obtain $\ell2$ covers at the optimal scale for real-valued function classes.1

Sources (1)

• [1]Uniform convergence at optimal rate and scale
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 12:04 AM

  For binary-valued classes, the Vapnik--Chervonenkis theorem characterizes uniform convergence by finiteness of the VC dimension.

  Uniform convergence asks how many random samples suffice for empirical averages to approximate expectations simultaneously over an entire class of functions.

Extractive summary: sentences quoted from the sources.