UniRAP: Towards Unified Part-level Physical Affordance Reasoning and Actionable Perception

Key points

• Vision-language perception has achieved impressive progress in aligning natural language with visual observations, yet grounding high-level semantics into part-level physical interaction remains challenging.1
• To predict executable contact geometry, we introduce a Unified Affordance Decoder (UAD), which jointly performs object detection, part-level affordance segmentation, and 4-DoF interaction pose estimation by leveraging intermediate segmentation features.1
• In addition, we propose a curriculum-based transfer training strategy that progressively adapts the model from general visual parsing to interaction-aware perception, improving data efficiency under limited textual annotations.1
• Experiments show that UniRAP achieves state-of-the-art performance on referring expression segmentation, affordance grounding, and interaction pose estimation, while maintaining strong spatiotemporal consistency in dynamic video scenarios.1

Sources (1)

• [1]UniRAP: Towards Unified Part-level Physical Affordance Reasoning and Actionable Perception
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 02:02 PM

  To address this gap, we propose UniRAP, a unified model for inferring part-level physical affordances and mapping language instructions to actionable geometric representations.

  Vision-language perception has achieved impressive progress in aligning natural language with visual observations, yet grounding high-level semantics into part-level physical interaction remains challenging.

Extractive summary: sentences quoted from the sources.