On the Limits of Representation-Based Test-Time Adaptation for Multimodal LLMs
While test-time adaptation (TTA) successfully mitigates these shifts in CLIP-like contrastive models, its application to generative multimodal large language models (MLLMs) remains underexplored and uniquely challenging due to their sequential token generation.
ProofPaper ↗
Key points
- Vision-language models suffer significant accuracy drops under common corruptions.
- In this work, we demonstrate that representation-based TTA fails to recover performance in MLLMs. Our experiments reveal that even when violating standard TTA assumptions by leveraging target data offline, sample-blind feature alignment recovers only marginal accuracy.
Sources (1)
- [1]On the Limits of Representation-Based Test-Time Adaptation for Multimodal LLMsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 09:50 AM
While test-time adaptation (TTA) successfully mitigates these shifts in CLIP-like contrastive models, its application to generative multimodal large language models (MLLMs) remains underexplored and uniquely challenging due to their sequential token generation.
Vision-language models suffer significant accuracy drops under common corruptions.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 14, 2026Text-Informed Training Data Filtering for Enhanced Virtual Flow Meter Performance
- Oct 14, 2026Who's Behind the Text? Robust Model Authorship Attribution
- Oct 14, 2026ReaDy: Recurrent States as Dynamic Context for Speculative Drafting in Hybrid Linear Attention Large Language Models
- Oct 13, 2026Gemini 4 Argon for Robotics
- Oct 13, 2026EmoGenie: Benchmarking and Evaluating Expressed and Evoked Emotion in T2I Generation with MLLMs
- Oct 13, 2026SpecCloak: Probing Harmful Code Generation through Engineering Specifications