Interaction Latents as Actions for Contact-Rich Manipulation

Key points

• Contact-rich manipulation requires robots to coordinate not only where to move, but also how contact and force should evolve throughout an interaction.1
• Yet most robot policies represent actions primarily as motion trajectories, using tactile signals as inputs or auxiliary prediction targets without explicitly encoding contact dynamics in the action representation.1
• We learn these latents through visually conditioned joint action-tactile reconstruction with a time-conditioned implicit decoder.1
• For offline improvement, we learn a tactile-conditioned expert reference model from interaction latents encoded from expert demonstrations.1

Sources (1)

• [1]Interaction Latents as Actions for Contact-Rich Manipulation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 05:59 PM

  We propose ILA (Interaction Latents as Actions), which uses decodable interaction latents that jointly capture the evolution of motion, contact, and force application as policy action units, providing a shared representation for action generation, evaluation, and offline improvement.

  Contact-rich manipulation requires robots to coordinate not only where to move, but also how contact and force should evolve throughout an interaction.

Extractive summary: sentences quoted from the sources.