Posterior Frequencies and the Cost of Not Fitting: A Correction to a Memorization Lower Bound
Feldman's memorization lower bound assigns a frequency coefficient based on how often a point appears in the training sample.
ProofPaper ↗
Key points
- We give an exact three-point counterexample to its conditional and averaged forms, using independent fair labels and a permutation-equivariant learner.
- The priors vary with sample size, the offending profiles become rare, and both quantities vanish.
- Retaining the original model, we obtain an exact replacement using full-sample posterior probabilities.
- We characterize the largest multiplicity-only coefficients valid for unrestricted learners.
Sources (1)
- [1]Posterior Frequencies and the Cost of Not Fitting: A Correction to a Memorization Lower BoundarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 04:04 PM
Feldman's memorization lower bound assigns a frequency coefficient based on how often a point appears in the training sample.
We give an exact three-point counterexample to its conditional and averaged forms, using independent fair labels and a permutation-equivariant learner.
Extractive summary: sentences quoted from the sources.