A Small Model Asks Better: Fine-Tuning Query Clarification for Retrieval-Augmented Generation

Key points

• We consider multi-turn grounded conversational scenarios where a user query is either ill-formed or ambiguous in the context of the prior dialog or, more interestingly, it may be well-formed but when posed against retrieved documents there are multiple relevant interpretations.1
• The task is to recognize both such cases and then generate an appropriate clarification request to the user.1
• We present ClariEdge, a new dataset of approximately 3,900 examples that include such ambiguous queries along with negative examples that do not require clarification.1
• We also show that a lightweight fine-tuned LoRA achieves 96.6% classification accuracy and outperforms prompted baselines of many larger models on both classification and quality of clarification generation.1

Sources (1)

• [1]A Small Model Asks Better: Fine-Tuning Query Clarification for Retrieval-Augmented Generation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 10:33 PM

  Query clarification is essential for retrieval-augmented generation (RAG) systems that face ambiguous user queries.

  We consider multi-turn grounded conversational scenarios where a user query is either ill-formed or ambiguous in the context of the prior dialog or, more interestingly, it may be well-formed but when posed against retrieved documents there are multiple relevant interpretations.

Extractive summary: sentences quoted from the sources.