Why Do VLMs Fail at Remote Physiological Measurement? Rethinking with Physiological Grounding and Numerical Alignment
Video-based remote physiological measurement (RPM) has broad applications, while Vision-Language Models (VLMs) provide a promising opportunity to enable end-to-end physiological monitoring for future human-aware intelligent systems.
ProofPaper ↗
Key points
- However, existing RPM methods rely on task-specific models and cannot be seamlessly integrated with the multimodal understanding and interaction capabilities of VLMs. We revisit RPM with VLMs and provide empirical evidence for two major bottlenecks: generic visual semantic representations fail to preserve fine-grained physiological cues; while discrete token prediction in autoregressive generation is inherently misaligned with continuous numerical measurement.
- To address these challenges, we propose VITAL (Visual-physiological Token Alignment for Language Models), a visual-physiological token alignment framework for RPM and preserving general vision tasks.
- VITAL retains the original visual pathway for semantic understanding and introduces an independent physiological pathway to jointly model semantic and physiological information.
- Moreover, we propose a structured numerical alignment approach to improve the reliability of generative heart rate estimation by enhancing numerical relationship modeling.
Sources (1)
- [1]Why Do VLMs Fail at Remote Physiological Measurement? Rethinking with Physiological Grounding and Numerical AlignmentarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 06:50 PM
Video-based remote physiological measurement (RPM) has broad applications, while Vision-Language Models (VLMs) provide a promising opportunity to enable end-to-end physiological monitoring for future human-aware intelligent systems.
However, existing RPM methods rely on task-specific models and cannot be seamlessly integrated with the multimodal understanding and interaction capabilities of VLMs. We revisit RPM with VLMs and provide empirical evidence for two major bottlenecks: generic visual semantic representations fail to preserve fine-grained physiological cues; while discrete token prediction in autoregressive generation
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 2026Run fast, private agents on-device with LFM2.5-24B-A2B
- Oct 13, 2026Gemini 4 Argon for Robotics
- Oct 13, 2026inclusionAI/Ling-3.1-flash
- Oct 13, 2026Hybrid Reasoning Agents
- Oct 13, 2026Fish Audio Drama 3 Technical Report
- Oct 12, 2026Microsoft’s new AI chip makes it less reliant on Nvidia’s GPUs