Revealing and Bridging the Physical Executability Gap in Vision-Language Navigation
We show that this assumption can fail: permissive contact models in current simulators may mask physically infeasible actions, creating a physical executability gap between semantic competence and embodied realization.
ProofPaper ↗
Key points
- Continuous vision-and-language navigation (VLN) is typically evaluated by goal-reaching success, assuming it reflects an agent's ability to execute language-grounded behavior.
- To bridge this gap, we introduce Visual-Semantic Traversability Alignment (VeSTA), a framework that aligns VLN policies with physically executable futures.
- We define visual-semantic traversability as the ability to predict instruction-conditioned trajectories that are semantically valid, geometrically feasible, and physically robust.
- Experiments on R2R and RxR reveal substantial dependence of existing VLN agents on simulator contact semantics.
Sources (1)
- [1]Revealing and Bridging the Physical Executability Gap in Vision-Language NavigationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 02:36 AM
We show that this assumption can fail: permissive contact models in current simulators may mask physically infeasible actions, creating a physical executability gap between semantic competence and embodied realization.
Continuous vision-and-language navigation (VLN) is typically evaluated by goal-reaching success, assuming it reflects an agent's ability to execute language-grounded behavior.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026PVBench: Quantifying Physical Plausibility in Video Generation with Object-Level Physical Measurements
- Oct 19, 2026Remembering Has a Price: Learning Memory Worth under Resource Constraints
- Oct 19, 2026VerifierBench: Benchmarking the Generation of Executable Verifiers for LLM Agents
- Oct 18, 2026Editing as Language: Teaching LLM Agents a Grammar for Native Context Editing
- Oct 18, 2026SoK: PIAtlas: Decomposing the Unbounded Prompt Injection Attack Space
- Oct 17, 2026Pair-Aware Optimization: Coordinating Exploration and Refinement for Jailbreaking LLM Agents