Camera-Noise Residuals for Face-Swap Detection: Redundant, Not Complementary, and Why

Key points

• We test, on FaceForensics++, whether a Noiseprint++ residual channel carries information complementary to an RGB Xception backbone for face-swap detection.1
• A three-model ablation (RGB-only, residual-only, late-fusion) shows that fusion does not improve over RGB alone and that the residual branch alone is near chance.1
• A seven-level bottleneck diagnostic localizes the cause: the noise maps do carry a discriminative signal, but it is statistical---carried by the per-sample first and second moments (mean, variance, energy) of the residual---and the per-sample InstanceNorm layer placed at the noise-branch input, following the TruFor template, standardizes exactly those moments away (five-fold cross-validated AUC drops from $0.747$ to $0.554$).1
• We conclude that, on this manipulation distribution, the noise residual is redundant with RGB rather than complementary, and we give concrete guidance for practitioners adopting noise-residual fusion for face-swap detection.1

Sources (1)

• [1]Camera-Noise Residuals for Face-Swap Detection: Redundant, Not Complementary, and Why
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 10:55 AM

  Fusing a learned camera-noise fingerprint with an RGB appearance backbone is an appealing route to generator-independent deepfake detection, because the noise residual is grounded in image-formation physics rather than in the texture statistics of a particular generator.

  We test, on FaceForensics++, whether a Noiseprint++ residual channel carries information complementary to an RGB Xception backbone for face-swap detection.

Extractive summary: sentences quoted from the sources.