Rendering-Free Lookahead for Question-Guided Active Vision

Key points

• Active robot vision requires controlling the camera to reveal task-relevant information that is hidden from the current viewpoint.1
• For viewpoint-dependent question answering, the challenge is to select camera motions that expose the visual evidence needed to answer the question.1
• At training, a privileged teacher renders candidate future views in 3D Gaussian Splatting (3DGS) scenes and uses a frozen VLM to compute one- and two-step answerability targets.1
• These results support learning camera-control policies from privileged visual lookahead for viewpoint-dependent question answering.1

Sources (1)

• [1]Rendering-Free Lookahead for Question-Guided Active Vision
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 12:45 AM

  We quantify this usefulness as answerability, a VLM's estimate that a view suffices to answer the question, and present Rendering-Free Lookahead (RFL), a viewpoint-selection policy that ranks candidate camera motions by predicted future answerability.

  Active robot vision requires controlling the camera to reveal task-relevant information that is hidden from the current viewpoint.

Extractive summary: sentences quoted from the sources.