Analyzing performance of Retrieval Augmented Generation models for Question Answering Tasks

Key points

• The cultural question and answering systems of South Asia, in particular, present a specific challenge since the region is home to the vast number of languages and traditions, which are closely interrelated.1
• Despite major architectural advances in LLMs, claiming significant improvements, challenges remain when answering these region specific questions in native languages.1
• In this study, we aim to extract high-performing factors to guide the development of optimized hybrid RAG systems.1
• We also examine how LLM performance changes when RAG methods are integrated into the question-answering pipeline in comparison with non-contexted and contexted answering and, evaluate whether the introduction of RAG benefits popular or non-popular culturally grounded questions more.1

Sources (1)

• [1]Analyzing performance of Retrieval Augmented Generation models for Question Answering Tasks
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 04:14 PM

  Retrieval-Augmented Generation (RAG) models have become almost a cornerstone of open-domain question answering tasks; in which the quality of answers generated in the process, greatly relies on the effectiveness of retrieval mechanisms.

  The cultural question and answering systems of South Asia, in particular, present a specific challenge since the region is home to the vast number of languages and traditions, which are closely interrelated.

Extractive summary: sentences quoted from the sources.