SenseAlign: Understanding Vision and Touch through Language at Test Time
Tactile-visual-language models combine vision and touch through a shared language space to recognize objects and materials.
ProofPaper ↗
Key points
- In real-world deployment, changes in lighting, contact quality or sensor hardware can shift visual or tactile inputs and degrade recognition performance.
- In this setting, we find that comparing accumulated observations with the text embeddings of candidate classes reveals modality reliability differences that confidence can miss.
- Refining features against competing classes also improves recognition beyond geometric alignment alone.
- These findings motivate SenseAlign, which uses a common semantic reference for test-time modality weighting and feature correction.
Sources (1)
- [1]SenseAlign: Understanding Vision and Touch through Language at Test TimearXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 04:50 PM
Tactile-visual-language models combine vision and touch through a shared language space to recognize objects and materials.
In real-world deployment, changes in lighting, contact quality or sensor hardware can shift visual or tactile inputs and degrade recognition performance.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Context-Aware Prompt Injection Detection via Small Language Models in RAG Pipelines: nokast-secureRAG
- Oct 20, 2026Voxtral 3
- Oct 20, 2026PAR: Progress-Aligned Recurrence for Length Generalization in State Tracking
- Oct 19, 2026XGLASS: X-Ray, Gaze and Language Assisted Slice Sequencing for CT Volumes
- Oct 19, 2026TokenGate: Adapting LLMs for Fast and Memory-Efficient Prefill with Learned Token Pruning
- Oct 19, 2026Competing Learning in Hybrid Architectures