Reliability-Aware Multimodal Sentiment Prediction Under Local Modality Missingness
Multimodal sentiment models are usually evaluated on complete utterances, yet streams often lose a contiguous span of text, audio, or video.
ProofPaper ↗
Key points
- We study local missingness on aligned CMU-MOSEI features.
- Our reliability-aware network (RM-Net) keeps the original temporal positions, represents unobserved positions with a learned token, outputs bounded per-modality intensity contributions gated by observation quality, and is trained with dual-view masking of contiguous segments.
- We fuse five RM-Net seeds with a LoRA-adapted BERT text expert, using a fusion weight that depends only on the visible fraction of content tokens.
- For explanations, exact Shapley attributions over the eight modality subsets and position-level occlusion show that top-ranked positions reduce the predicted-class probability faster than random positions (AUC gap 0.113 on validation).
Sources (1)
- [1]Reliability-Aware Multimodal Sentiment Prediction Under Local Modality MissingnessarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 05:36 PM
Multimodal sentiment models are usually evaluated on complete utterances, yet streams often lose a contiguous span of text, audio, or video.
We study local missingness on aligned CMU-MOSEI features.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 14, 2026Tessera-27B-B-CJK, Zellige AIs new finetune of Qwen with novel approaches to how we think about token savings
- Oct 14, 2026NVIDIA/TensorRT-LLM v1.4.0rc0
- Oct 14, 2026ReGRA: Residual-Gradient Adaptation for Complementary Low-Rank Fine-Tuning
- Oct 14, 2026Train LoRA directly on Quantized GGUFs, no PyTorch, no Parent(FP16) needed, just your GGUF and the data
- Oct 13, 2026Comfy-Org/ComfyUI v0.40.0
- Oct 13, 2026unslothai/unsloth v0.1.907-beta: Large Performance Gains + More Browser Use