Sample-Optimal Multi-Group Learning with an ERM Oracle

Key points

• In multi-group learning, a single predictor must perform nearly as well as the best hypothesis in a class on every group in a rich, possibly infinite and overlapping, collection of subpopulations.1
• We ask whether the optimal rate can be attained by a learner that accesses the classes only through an empirical risk minimization (ERM) oracle.1
• We also show a matching lower bound of $Ω((dG+dH)/\varepsilon^2)$ samples for every learner even with unlimited computation, so oracle efficiency costs no additional samples.1
• In this case, an ERM oracle for the hypothesis class alone suffices, with $\tilde{O}((dH+\log k)/\varepsilon^2)$ samples in the agnostic setting and $\tilde{O}((dH+\log k)/\varepsilon)$ samples when every group has a hypothesis that is correct on it.1

Sources (1)

• [1]Sample-Optimal Multi-Group Learning with an ERM Oracle
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 11, 08:56 PM

  We give an algorithm that uses $\tilde{O}((d_G+d_H)/\varepsilon^2)$ samples and polynomially many calls to a weighted ERM oracle over pairs of a group and a hypothesis, where $d_G$ and $d_H$ are the VC dimensions of the group and hypothesis classes and $\varepsilon$ bounds the excess risk on every group, weighted by its probability.

  In multi-group learning, a single predictor must perform nearly as well as the best hypothesis in a class on every group in a rich, possibly infinite and overlapping, collection of subpopulations.

Extractive summary: sentences quoted from the sources.