Neural Expansion: A Unified Mechanism for How Deep Neural Networks Generalize
We introduce Neural Expansion, a unified geometric framework that characterizes generalization through the expansion of directionally-selective ray coverage in the input space.
ProofPaper ↗
Key points
- Deep neural networks demonstrate remarkable generalization despite being highly over-parameterized, challenging traditional statistical learning theories.
- While several theoretical paradigms such as margin-based bounds, PAC-Bayesian analyses, and neural tangent kernels provide global explanations, they often lack mechanistic insight into how generalization emerges from training dynamics.
- Using Generalization Intervals (GIs), we quantify direction-specific regions around training samples where the model's predictions remain stable.
- By connecting input-space curvature to predictive behavior, our framework provides a mechanistic foundation that complements existing global theories of deep learning.
Sources (1)
- [1]Neural Expansion: A Unified Mechanism for How Deep Neural Networks GeneralizearXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 04:38 PM
We introduce Neural Expansion, a unified geometric framework that characterizes generalization through the expansion of directionally-selective ray coverage in the input space.
Deep neural networks demonstrate remarkable generalization despite being highly over-parameterized, challenging traditional statistical learning theories.
Extractive summary: sentences quoted from the sources.