Generalizing Multimodal Foundational Models in Context-Driven Class Incremental Learning
Motivated by this, we propose a framework that generalizes frozen MFMs to audio-visual CIL.
ProofPaper ↗
Key points
- Class-Incremental Learning (CIL) aims to learn new classes continuously without forgetting the previously learned ones.
- Recent works in CIL have significantly advanced across various modalities; however, at the intersection of audio-visual modality, this problem remains underexplored.
- While representations from multimodal foundational models (MFMs) encapsulate rich static priors, our preliminary analysis shows that these representations struggle in incremental settings.
- Specifically, we leverage their dense audio and visual representations and employ an attention-based contextual guidance module, in which the audio representations serve as task-stable contextual guidance for the visual representations, enabling effective class recognition in the incremental setting.
Sources (1)
- [1]Generalizing Multimodal Foundational Models in Context-Driven Class Incremental LearningarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 09:13 PM
Motivated by this, we propose a framework that generalizes frozen MFMs to audio-visual CIL.
Class-Incremental Learning (CIL) aims to learn new classes continuously without forgetting the previously learned ones.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026Bridged Self-Distillation for On-Policy Learning
- Oct 19, 2026Selective Transfer through Sequence Distillation: From Benchmark Knowledge to Backdoor Behaviour
- Oct 19, 2026Functional Cross-layer Experts Routing for One-step Diffusion Distillation
- Oct 19, 2026ReDOA: Real-World Distillation via Observation Alignment for Image-Based Manipulation Policy Learning
- Oct 18, 2026Multimodal-Conditioned Latent Diffusion Models for Spatiotemporal Brain MRI Forecasting
- Oct 18, 2026ClinSDT: LLM-Encoded Clinical Semantic Guidance for Sepsis Treatment via Decision Transformer