SurgCast: Action-Conditioned Future Skeletons for Controllable Surgical World Models
We introduce SurgCast, which predicts future instrument skeletons from current structure, robot state, and future actions, then uses the forecast to control video generation.
ProofPaper ↗
Key points
- Surgical world models must turn prospective robot actions into visually faithful, structurally consistent instrument motion.
- Dual-path injection combines skeleton geometry with derived, instrument-specific motion descriptors through geometric residuals and conditioning modulation.
- Ablations show that future actions improve skeleton prediction and that combining both injection paths performs best across all six metrics under predicted skeletons.
- These results support using predicted articulated structure and its motion semantics as a control interface for surgical video prediction.
Sources (1)
- [1]SurgCast: Action-Conditioned Future Skeletons for Controllable Surgical World ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 11, 02:21 PM
We introduce SurgCast, which predicts future instrument skeletons from current structure, robot state, and future actions, then uses the forecast to control video generation.
Surgical world models must turn prospective robot actions into visually faithful, structurally consistent instrument motion.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 11, 2026StreamingWAM: Streaming World Action Modeling via Chunk-Wise Staircase Flow Matching
- Oct 11, 2026A Plausible Future Is Not a Causal Future: Can Video Models Generate How the World Unfolds?
- Oct 11, 2026ObsSpec: Accelerating Agentic Reinforcement Learning with Observation Speculation
- Oct 10, 2026MORE: Learning Motion-Oriented Registers for Driving World Models and Planning
- Oct 9, 2026Beyond Successful Robot Demonstrations: Learning from Human Interactions and Modeling Failure Dynamics
- Oct 9, 2026Latent World Action Model for GUI Agents