DIME: Scaling Facial Representation Learning via Differential Masked Autoencoding

Key points

• Scaling up data and model capacity has driven substantial progress in visual representation learning.1
• We attribute this to a scaling bottleneck: these tasks model each image's appearance and are dominated by identity, pose, and illumination, so subtle within-identity facial differences contribute little to the training signal.1
• DIME reconstructs both masked images in an identity-matched pair and, through an Explicit Differential Direction Supervision (EDDS) objective, aligns the direction of their reconstructed difference with the true difference, while source isolation makes each reconstruction depend only on its own image.1
• We pretrain DIME on 20 million high-resolution facial images consolidated from existing face datasets.1

Sources (1)

• [1]DIME: Scaling Facial Representation Learning via Differential Masked Autoencoding
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 03:57 PM

  We introduce DIME, a differential masked autoencoder for large-scale facial representation learning.

  Scaling up data and model capacity has driven substantial progress in visual representation learning.

Extractive summary: sentences quoted from the sources.