MAIA: Anticipating Interaction Anomalies in Contact-Rich Manipulation via Multimodal World Models

Key points

• Reliable contact-rich manipulation requires detecting execution deviations early enough for corrective action.1
• We address this problem with MAIA, a multimodal framework for anomaly anticipation.1
• Its vision-action-force world model learns nominal interaction dynamics and predicts the visual and force responses of planned actions, while a lightweight anomaly head predicts impending interaction anomalies.1
• Moreover, using these predictions for anomaly-aware action selection improves end-to-end task success from 52.6% to 61.5% without retraining the underlying manipulation policy.1

Sources (1)

• [1]MAIA: Anticipating Interaction Anomalies in Contact-Rich Manipulation via Multimodal World Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 11:11 AM

  To enable systematic evaluation, we introduce CRIA, a real-world multimodal dataset of 2,852 trajectories spanning 18 contact-rich manipulation tasks, with frame-level annotations of anomaly onset and evolution.

  Reliable contact-rich manipulation requires detecting execution deviations early enough for corrective action.

Extractive summary: sentences quoted from the sources.