RiSE: Residual Subspace Expert for Generalizable Text-Centric Image Forgery Localization

Key points

• As AI-assisted image editing becomes increasingly prevalent, Text-Centric Image Forgery Localization (TFL) is essential for protecting trust in financial and legal records.1
• However, we observe that existing TFL detectors predominantly rely on Full-Parameter Fine-Tuning (FPFT) and fail to generalize to unseen forgery types because FPFT can induce low-rank feature collapse and overfit training artifacts.1
• To address these challenges, we propose RiSE, a Residual Subspace Expert model that preserves pre-trained priors by freezing principal SVD components and adapting only the residual subspaces with a ViT-only segmentor to avoid feature collapse.1
• At inference, we introduce Latent Perturbation Confidence (LPC), which selects the most confident expert by analytically computing the latent-perturbed output.1

Sources (1)

• [1]RiSE: Residual Subspace Expert for Generalizable Text-Centric Image Forgery Localization
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 05:53 AM

  RiSE introduces forgery-aware residual experts, where each expert is trained independently on a training subset with a particular forgery type, and we show that cross-domain localization performance improves consistently as more experts are added.

  As AI-assisted image editing becomes increasingly prevalent, Text-Centric Image Forgery Localization (TFL) is essential for protecting trust in financial and legal records.

Extractive summary: sentences quoted from the sources.