CaptionFuse: Training-Free Composed Product Retrieval Via VLM Intent Captioning and Weighted Embedding Fusion

Key points

• Composed Image Retrieval (CIR) retrieves products matching a reference image modified by natural language---a core capability for eCommerce search.1
• Existing training-free methods fuse CLIP embeddings via arithmetic, but we identify a fundamental limitation: user text modifications are attribute-sparse---a user who types "make it red with shorter sleeves" specifies what to change but not what to keep (the floral pattern, the V-neck, the fit-and-flare silhouette).1
• Our method, CaptionFuse, requires no task-specific training and is model-agnostic.1
• A controlled experiment confirms that attribute content--- not linguistic style---drives the gains: imperative and descriptive captions with identical attributes achieve equal R@10, while both outperform raw modifications by +6.5 points.1

Sources (1)

• [1]CaptionFuse: Training-Free Composed Product Retrieval Via VLM Intent Captioning and Weighted Embedding Fusion
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 04:18 PM

  We introduce language-space fusion: a Vision-Language Model (VLM) jointly interprets the reference image and modification to generate a visually grounded product-title caption---e.g., "Red floral fit-and-flare dress with short sleeves and V-neck"---that fills in the missing attributes from the image.

  Composed Image Retrieval (CIR) retrieves products matching a reference image modified by natural language---a core capability for eCommerce search.

Extractive summary: sentences quoted from the sources.