RoPE-Flow: Monocular Vision-based Robot Pose Estimation via 2D Graph-Conditioned 3D Flow Matching

Key points

• Camera-to-robot pose estimation from a monocular image is significant for vision-based robot navigation and control.1
• From the predicted 2D heatmaps, we propose a Keypoint-centered Feature Extractor to obtain positional features along with contextual representations from keypoints-centered multi-scale patches.1
• Moreover, we introduce Graph-guided Condition Generator, which embeds a learnable adjacency map to generate condition for flow matching based on correlation among keypoints.1
• Besides, we establish a novel cross-scene benchmark to demonstrate the outstanding performance of RoPE-Flow on zero-shot cross-scene robot pose estimation.1

Sources (1)

• [1]RoPE-Flow: Monocular Vision-based Robot Pose Estimation via 2D Graph-Conditioned 3D Flow Matching
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 01:40 AM

  In this work, we propose RoPE-Flow, an end-to-end flow matching-based framework for camera-to-robot pose estimation.

  Camera-to-robot pose estimation from a monocular image is significant for vision-based robot navigation and control.

Extractive summary: sentences quoted from the sources.