WinARDet: Compressed Window Auto-Regression for Long-Horizon Vision-Language Grounding

Key points

• MLLM-based detectors formulate vision-language grounding as autoregressive coordinate generation, yet in dense scenes, they often collapse into a box repetition loop, repeatedly emitting identical boxes until the decoding budget is exhausted.1
• This new mechanism partitions a long answer sequence into consecutive generation windows and restricts full token-level autoregression to the current window only while compressing each previous window into a fixed set of latent tokens that condition subsequent decoding.1
• This design reduces the repeated exposure to raw historical coordinates that triggers the attention takeover, yet preserves cross-window dependencies essential for coherent multi-object generation.1
• WinARDet also generalizes across decoding strategies and output formats, including parallel box decoding and polygon generation, providing a practical paradigm for long-horizon vision-language grounding across scene densities.1

Sources (2)

• [1]WinARDet: Compressed Window Auto-Regression for Long-Horizon Vision-Language Grounding
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 02:09 AM

  We trace this failure to historical-answer attention takeover: Under full-history autoregression, repeated coordinate tokens in the growing answer history attract disproportionate attention, crowding out the attention sinks needed to attend to new visual evidence. % To break this loop, in this paper, we propose Compressed Window Auto-Regression (WinAR).

  MLLM-based detectors formulate vision-language grounding as autoregressive coordinate generation, yet in dense scenes, they often collapse into a box repetition loop, repeatedly emitting identical boxes until the decoding budget is exhausted.

• [2]WinARDet: Compressed Window Auto-Regression for Long-Horizon Vision-Language Grounding
  Hugging Face Daily Papers · Oct 14, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.