FoveaTree: Information-Density Trees for Query-Guided Visual Token Pruning in Video Large Language Models

Key points

• Video large language models (Video LLMs) encode each frame into hundreds of visual tokens, making video inference expensive.1
• Existing methods reduce this cost through similarity-based merging or attention-based selection but face two challenges: Hierarchy-blind Pruning, which overlooks regional compressibility and can discard critical evidence, and Attention Drift, where positional effects and attention sinks shift text-to-vision attention away from task relevance.1
• Second, Attention Tree Pruning uses the surviving units and inherited tree topology to refine text-to-vision attention in intermediate LLM layers, preserving relevant regional evidence while removing low-relevance content.1
• We evaluate FoveaTree on InternVL3.5-8B and Qwen3-VL-8B across five video question-answering benchmarks and one captioning benchmark at 50%, 30%, and 10% visual-token retention.1

Sources (1)

• [1]FoveaTree: Information-Density Trees for Query-Guided Visual Token Pruning in Video Large Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 01:11 PM

  We propose FoveaTree, a two-stage, training-free visual-token pruning framework.

  Video large language models (Video LLMs) encode each frame into hundreds of visual tokens, making video inference expensive.

Extractive summary: sentences quoted from the sources.