Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution
We introduce Video-RSI, a framework for recursive self-improvement in which a video understanding agent uses its own language model to revise its harness.
ProofPaper ↗
Key points
- Video understanding agents acquire evidence through an executable harness that controls what they observe and how they use those observations.
- Cost-aware harness evolution turns these diagnoses into reusable revisions and determines which revisions to retain by considering both answer accuracy and visual cost.
- Across our evaluation settings on video understanding benchmarks, the evolved agent improves accuracy while processing fewer frames and achieves competitive accuracy-efficiency trade-offs against existing video understanding agents.
- These results demonstrate the potential for video understanding agents to improve their own evidence acquisition and use through harness evolution.
Sources (1)
- [1]Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness EvolutionHugging Face Daily Papers · Sep 29, 12:00 AM
We introduce Video-RSI, a framework for recursive self-improvement in which a video understanding agent uses its own language model to revise its harness.
Video understanding agents acquire evidence through an executable harness that controls what they observe and how they use those observations.
Extractive summary: sentences quoted from the sources.