Semantic Latent Alignment for Flow Matching with Invertible Bi-Lipschitz Adapters

Key points

• We instead leave the encoder and decoder fixed and change only the coordinates between them, using an invertible bi-Lipschitz adapter aligned offline to a semantic teacher and frozen during class-conditional flow matching.1
• Invertibility preserves compatibility with the pretrained autoencoder, while the bi-Lipschitz constraint limits how far alignment can distort its latent geometry.1
• On two medical imaging datasets, constrained latent adaptation improves feature metrics, seed stability and counterfactual edit strength, while remaining competitive on downstream classification.1
• Our results suggest that carefully constrained post-hoc adaptation offers a parameter-efficient and tuning-free alternative to jointly modifying the representation during generative training.1

Sources (1)

• [1]Semantic Latent Alignment for Flow Matching with Invertible Bi-Lipschitz Adapters
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 06:19 AM

  Pretrained latent spaces trade compression, reconstruction, and the learnability of the coordinates a downstream generator has to work in, which complicates adapting these spaces to different domains or tasks: continued training is expensive, and without a new alignment objective it often fails to improve downstream generation.

  We instead leave the encoder and decoder fixed and change only the coordinates between them, using an invertible bi-Lipschitz adapter aligned offline to a semantic teacher and frozen during class-conditional flow matching.

Extractive summary: sentences quoted from the sources.