Progressive Subset Selection and Cascaded Consistency for Unsupervised Vision-Language Model Adaptation

Key points

• Unsupervised adaptation provides a practical way to improve target-domain performance without labeled data, but it remains challenging because pseudo-labels generated by the model can be unreliable and no labeled validation set is available for model selection or early stopping.1
• We propose a simple framework for unsupervised PVLM adaptation that combines progressive margin-based subset selection with cascaded consistency learning.1
• Cascaded consistency learning further exploits unlabeled samples by using temporally smoothed teacher predictions on weakly augmented views to supervise student predictions on strongly augmented views, with confidence filtering to reduce the influence of uncertain pseudo-labels.1
• Experiments across multiple recognition benchmarks with both CLIP and SigLIP backbones show consistent improvements over zero-shot baselines and competitive performance against recent unsupervised PVLM adaptation methods.1

Sources (1)

• [1]Progressive Subset Selection and Cascaded Consistency for Unsupervised Vision-Language Model Adaptation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 02:30 AM

  Pretrained Vision-Language Models (PVLMs) achieve strong zero-shot recognition but often suffer performance degradation under target-domain shift.

  Unsupervised adaptation provides a practical way to improve target-domain performance without labeled data, but it remains challenging because pseudo-labels generated by the model can be unreliable and no labeled validation set is available for model selection or early stopping.

Extractive summary: sentences quoted from the sources.