Temporal Specification Guidance for Text-to-Video Generation

Key points

• Text-to-video models often generate the visual concepts requested in natural language prompts but fail to satisfy their intended ordering, timing, and persistence requirements.1
• TSG scores traces of visual concept evidence using differentiable quantitative semantics and uses the score's gradients to construct a specification-guided denoising direction.1
• We instantiate this framework with video-text attention traces, efficient differentiable scores for Metric Temporal Logic, and gradient-derived attention biases.1
• These results demonstrate the potential of temporal specifications and differentiable satisfaction objectives as a flexible interface for controlling video generation.1

Sources (1)

• [1]Temporal Specification Guidance for Text-to-Video Generation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 07:09 PM

  We propose representing these requirements as specifications in a formal Temporal Logic, and introduce Temporal Specification Guidance (TSG), a training-free inference-time framework that turns temporal specifications into executable objectives for guiding video generation during sampling.

  Text-to-video models often generate the visual concepts requested in natural language prompts but fail to satisfy their intended ordering, timing, and persistence requirements.

Extractive summary: sentences quoted from the sources.