Sufficient Frame Subsets for Video Understanding
We introduce Sufficient Frame Subsets: compact sets of frames in which all other frames are masked, yet each subset independently preserves the model's prediction.
ProofPaper ↗
Key points
- Understanding when during a video a model makes its critical observations is essential for interpreting video-understanding systems.
- We evaluate our method on two complementary benchmarks, UCF101 and Something-Something v2, using both CNN- and transformer-based architectures.
- Our results show that the proposed algorithm identifies highly compact and effective frame subsets for explaining model decisions.
- Finally, the extracted frame subsets help us analyze model failure cases, reveal which aspects of a video the model has learned to emphasize, and identify opportunities for improving future training procedures.
Sources (1)
- [1]Sufficient Frame Subsets for Video UnderstandingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 07:47 AM
We introduce Sufficient Frame Subsets: compact sets of frames in which all other frames are masked, yet each subset independently preserves the model's prediction.
Understanding when during a video a model makes its critical observations is essential for interpreting video-understanding systems.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 17, 2026deepseek-ai/DeepSeek-V4.1-Flash-1017
- Oct 16, 2026nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-MXFP8
- Oct 16, 2026Welcome Beam: a 501B open model you can own, customize, and scale
- Oct 15, 2026Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint
- Oct 15, 2026When Plans Change, Remember: Replanning-Guided Visual Memory for Vision-Language-Action Models
- Oct 15, 2026huggingface/transformers v5.20.0: Release v5.20.0