IsoVRBench: Video Reward Modeling Without Human Feedback

Key points

• Video reward models are trained and evaluated on human preferences between videos that often differ simultaneously on properties such as prompt alignment, visual fidelity, motion, and generator identity.1
• As a result, aggregate accuracy does not reveal which video properties a reward model can reliably distinguish.1
• We construct preference pairs by controlled interventions that change one property of a generated video while holding the rest fixed.1
• Using the synthetic isolated intervation pairs as training data leads to 1) IsoVRM, a reward model outperforming its counterpart trained with a larger human labeled preference dataset by 3.8 points on VBench-2.0, 2) IsoWan, the video generator post-trained with IsoVRM, outperforming its human-data counterpart in human preferences, and 3) gains that stack when combined with human data.1

Sources (1)

• [1]IsoVRBench: Video Reward Modeling Without Human Feedback
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 06:03 PM

  We propose IsoVRBench, a video generation reward modeling benchmark of 8,000 such pairs across eight interventions, and show that state-of-the-art models separate broad quality differences between generators but miss important features such as temporal order.

  Video reward models are trained and evaluated on human preferences between videos that often differ simultaneously on properties such as prompt alignment, visual fidelity, motion, and generator identity.

Extractive summary: sentences quoted from the sources.