GraftOOD: Grafting Out-of-Distribution Exemplars into Advantage-Weighted Rectification for Video Reward Optimization

Key points

• We propose GraftOOD, which replaces part of the online rollouts in selected prompt groups with out-of-distribution video exemplars.1
• A shared velocity-regression objective combines group-relative reward rectification with sequence-level exemplar supervision, requiring neither exemplar generation trajectories nor an additional supervised-loss coefficient.1
• On Wan2.1-1.3B, GraftOOD raises Dynamic Degree from 4.86 to 44.41 relative to reward-only FlowAWR, while HPSv3 decreases from 8.10 to 7.32 and VisionReward remains close.1
• These results support external exemplar supervision as a complement to scalar reward feedback, without equating higher measured dynamics with coherent, visually plausible motion in the generated videos.1

Sources (1)

• [1]GraftOOD: Grafting Out-of-Distribution Exemplars into Advantage-Weighted Rectification for Video Reward Optimization
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 12:06 PM

  Video reward optimization can increase proxy rewards while reducing video dynamics, a mismatch we identify as motion collapse, including under a video-level motion-quality reward.

  We propose GraftOOD, which replaces part of the online rollouts in selected prompt groups with out-of-distribution video exemplars.

Extractive summary: sentences quoted from the sources.