ReferFlow: Beyond Final-Target Localization for Complex Referring Expressions

Key points

• Complex queries often involve intermediate visual references with different dependencies on prior grounding, but final-only evaluation leaves this process unexamined.1
• To address this gap, we formulate trajectory-level REC, in which models receive an ordered sequence of grounding steps constructed from a single complex query and localize the target regions associated with each step.1
• We instantiate this setting in ReferFlow-Bench, which characterizes structured region transitions through four trajectory types, namely Refinement, Shift, Reset, and Mixture.1
• We further propose ReferFlow, which decouples interleaved text-latent trajectory generation from coordinate decoding.1

Sources (1)

• [1]ReferFlow: Beyond Final-Target Localization for Complex Referring Expressions
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 03:29 PM

  Multimodal Large Language Models have made significant progress in referring expression comprehension (REC), yet most REC benchmarks still focus on final-target localization.

  Complex queries often involve intermediate visual references with different dependencies on prior grounding, but final-only evaluation leaves this process unexamined.

Extractive summary: sentences quoted from the sources.