StreamEval: Evaluating Interactive Streaming Video Generation Models

Key points

• As streaming video generation advances, enabling real-time interaction with generated video is becoming increasingly important.1
• StreamEval comprises 300 scripted test cases lasting 15-300s, each specifying initial assets, incrementally arriving multimodal requests, and explicit requirements for state changes and content preservation.1
• Evaluating 37 distinct models across 300 test cases, we find that high visual quality and identity retention do not necessarily coincide with reliable request execution or sustained state consistency.1
• These findings identify reliable request execution under timing and preservation constraints as a central challenge for advancing omni-modal real-time interactive systems.1

Sources (2)

• [1]StreamEval: Evaluating Interactive Streaming Video Generation Models
  Hugging Face Daily Papers · Oct 20, 12:00 AM

  We introduce StreamEval, a unified benchmark for interactive streaming video generation spanning five tasks: avatar dialogue, avatar performance, world generation, instruction editing, and reference editing.

  As streaming video generation advances, enabling real-time interaction with generated video is becoming increasingly important.

• [2]StreamEval: Evaluating Interactive Streaming Video Generation Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 03:47 PM · same content

Extractive summary: sentences quoted from the sources.