MM-LeJEPA: Correlated Gaussian Regularization for Multimodal Joint-Embedding Predictive Architectures

Key points

• LeJEPA derives an isotropic Gaussian embedding target by minimizing, under its task assumptions, a Fisher information criterion that controls the task-averaged squared prediction bias.1
• This enables unimodal self-supervised learning without reconstruction decoders, exponential-moving-average teachers, or contrastive negatives.1
• Under explicit gradient-moment, correspondence, and symmetry assumptions, we derive a correlated Gaussian target that minimizes a weighted Fisher information functional under a fixed total variance constraint.1
• An analytic transform adapts Sketched Isotropic Gaussian Regularization (SIGReg) to this target, yielding a two-term objective that combines distributional regularization with local-to-global prediction.1

Sources (1)

• [1]MM-LeJEPA: Correlated Gaussian Regularization for Multimodal Joint-Embedding Predictive Architectures
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 01:20 PM

  We introduce MM-LeJEPA by extending LeJEPA's task assumptions to multimodal learning.

  LeJEPA derives an isotropic Gaussian embedding target by minimizing, under its task assumptions, a Fisher information criterion that controls the task-averaged squared prediction bias.

Extractive summary: sentences quoted from the sources.