Ego2Contact: Predicting Touch from Egocentric Human Video for Contact-Rich Robot Manipulation

Key points

• Contact locations and interaction forces are critical for contact-rich robot manipulation but are often ambiguous from vision alone.1
• Recent predictors trained on egocentric recordings paired with tactile-glove measurements can predict this missing signal from video alone.1
• Anatomical region correspondences align varying robot tactile layouts with the predictor's canonical grid, turning robot measurements into fine-tuning targets and predictions into policy inputs.1
• Ego2Contact outperforms prior tactile predictors on human data, including unseen objects, and on robot data after adaptation.1

Sources (1)

• [1]Ego2Contact: Predicting Touch from Egocentric Human Video for Contact-Rich Robot Manipulation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 04:51 AM

  We present Ego2Contact, which learns a visual-tactile prior from human interactions and transfers it to robot hands.

  Contact locations and interaction forces are critical for contact-rich robot manipulation but are often ambiguous from vision alone.

Extractive summary: sentences quoted from the sources.