FrameGRPO: Incentivizing Video Reasoning via Multi-Frame Rollouts and Temporal Credit
Reinforcement learning with verifiable rewards, particularly Group Relative Policy Optimization (GRPO), has improved video reasoning in multimodal large language models.
ProofPaper ↗
Key points
- However, standard video GRPO varies only linguistic trajectories while conditioning every rollout in a group on the same sampled frames, which can produce homogeneous rewards and weaken group-relative learning when the shared visual evidence is insufficient or distracting.
- We propose FrameGRPO, a simple and effective reinforcement-learning framework that requires neither auxiliary networks nor changes to the model architecture.
- Extensive experiments across diverse video understanding and reasoning benchmarks demonstrate robust overall improvements over standard GRPO, highlighting temporal-view diversification as an effective mechanism for exploration and credit assignment in video MLLMs.
Sources (1)
- [1]FrameGRPO: Incentivizing Video Reasoning via Multi-Frame Rollouts and Temporal CreditarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 08:45 AM
Reinforcement learning with verifiable rewards, particularly Group Relative Policy Optimization (GRPO), has improved video reasoning in multimodal large language models.
However, standard video GRPO varies only linguistic trajectories while conditioning every rollout in a group on the same sampled frames, which can produce homogeneous rewards and weaken group-relative learning when the shared visual evidence is insufficient or distracting.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026Adaptive Decoding Methods for Token-Efficient Reinforcement Learning
- Oct 19, 2026Think, Compress, Answer: Reinforcement Learning for Multimodal Grounding via Reasoning Compression
- Oct 18, 2026Conformal Prediction-Enabled Sequence-level Policy Optimization for Reasoning Language Models
- Oct 18, 2026Think Visually Across Space and Time: Grounded Reasoning for Video Segmentation
- Oct 17, 2026ProbePO: Sparse Probes, Redistributed Credit for Critic-Free LLM Reinforcement Learning
- Oct 17, 2026Axiomatic World Modeling for Physics Reasoning