Learning Geometry-Aware Planning Representations for Monocular Flight with Cross-Modal Supervision

Key points

• Existing high-speed navigation systems for micro aerial vehicles (MAVs) commonly rely on depth cameras, stereo vision, or LiDAR to obtain explicit geometric information, while monocular cameras offer a lighter and lower-cost alternative.1
• However, monocular flight remains challenging because RGB images offer no direct metric cues and visual appearance varies considerably across environments.1
• Without explicitly reconstructing depth, the proposed method complements the downstream planning objective with feature-level geometric supervision by transferring the planning-relevant representation learned by a privileged depth expert to the RGB planner through cross-modal distillation.1
• We further use a 3D Gaussian Splatting (3DGS) simulator to render visual observations for data collection, paired RGB-D supervision, and simulation evaluation.1

Sources (1)

• [1]Learning Geometry-Aware Planning Representations for Monocular Flight with Cross-Modal Supervision
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 02:44 PM

  We propose a geometry-aware monocular visual planning framework that predicts trajectories from two image frames, their relative pose, and the MAV state.

  Existing high-speed navigation systems for micro aerial vehicles (MAVs) commonly rely on depth cameras, stereo vision, or LiDAR to obtain explicit geometric information, while monocular cameras offer a lighter and lower-cost alternative.

Extractive summary: sentences quoted from the sources.