Beyond Shortcut Learning: Hierarchical Spatial Prototype Regularization for Face Forgery Detection
Leveraging this insight, we propose Hierarchical Spatial Prototype (HSP) regularization, which explicitly models the forgery manifold by learning a structured prototype subspace over spatial features.
ProofPaper ↗
Key points
- Recent advances in generative models have enabled highly realistic face forgeries, yet generalizing deepfake detectors to unseen manipulation pipelines remains a major challenge.
- In this paper, we provide new evidence that naively fine-tuned vision models suffer from severe forensic representation collapse: their extracted cues are often entangled with unstable, dataset-specific artifacts, resulting in structurally fragile representations and poor cross-domain generalization.
- We further introduce prototype orthogonality and second-order assignment balance, which jointly preserve the geometric independence of latent forgery factors and promote high-effective-rank utilization of the forensic representation, thereby reducing reliance on dataset-specific shortcuts.
- These results demonstrate the practical value of HSP as an easy-to-adopt regularization module for enhancing deepfake detectors.
Sources (1)
- [1]Beyond Shortcut Learning: Hierarchical Spatial Prototype Regularization for Face Forgery DetectionarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 09:02 AM
Leveraging this insight, we propose Hierarchical Spatial Prototype (HSP) regularization, which explicitly models the forgery manifold by learning a structured prototype subspace over spatial features.
Recent advances in generative models have enabled highly realistic face forgeries, yet generalizing deepfake detectors to unseen manipulation pipelines remains a major challenge.
Extractive summary: sentences quoted from the sources.