GPT-6-Astra Understands Touch and Outperform Vision--Tactile--Language--Action Models

Key points

• General-purpose multimodal models have shown promise as direct robot action policies, yet whether they grasp the physics underlying real-world interactions and can apply that understanding to guide action remains underexplored.1
• We investigate this question using contact-rich manipulation as a representative testbed, examining tactile understanding as a concrete instance of this broader capability.1
• Contact-rich manipulation requires interpreting tactile feedback and adjusting actions after obstruction.1
• GPT-6-Astra combines this context with multimodal observations and interaction history to generate control commands and adjust actions through contact feedback.1

Sources (1)

• [1]GPT-6-Astra Understands Touch and Outperform Vision--Tactile--Language--Action Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 04:15 PM

  We present GPT-as-Tactile-Policy, which summarizes expert demonstrations into task guidance and tactile references for contact assessment and corrective action.

  General-purpose multimodal models have shown promise as direct robot action policies, yet whether they grasp the physics underlying real-world interactions and can apply that understanding to guide action remains underexplored.

Extractive summary: sentences quoted from the sources.