HorizonComposer: Spatiotemporally Consistent Driving Video Editing with Enriched Traffic Semantics

Key points

• However, driving scenes operate within a dense Operational Design Domain (ODD) governed by strict traffic semantics.1
• First, we construct a hybrid dataset by combining temporally aligned real-world videos with synthetic edits, balancing foundational scene structure with unambiguous appearance signals.1
• Finally, our model mitigates temporal shock by introducing thinking frames, a transition mechanism that seamlessly reconciles static image conditioning with dynamic video context.1
• Extensive experiments demonstrate that our method prevents the corruption of multi-actor dynamics, establishing state-of-the-art spatiotemporal consistency, yielding consistent user preference (39% relative improvement in content preservation and 12% in temporal consistency), and achieving a significant 5% gain in downstream road line segmentation and 7% gain in downstream multi-object detection.1

Sources (1)

• [1]HorizonComposer: Spatiotemporally Consistent Driving Video Editing with Enriched Traffic Semantics
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 06:51 PM

  Instruction-guided video generative models offer a scalable solution for stress-testing autonomous driving systems by simulating diverse environmental conditions.

  However, driving scenes operate within a dense Operational Design Domain (ODD) governed by strict traffic semantics.

Extractive summary: sentences quoted from the sources.