RAG-NIMBF: A Non-Invasive, Modular Benchmarking Framework for Automated RAG Configuration Analysis
Retrieval-Augmented Generation (RAG) has become the dominant paradigm for grounding the outputs of large language models (LLM) in external knowledge.
ProofPaper ↗
Key points
- However, practitioners are faced with a vast configuration space comprising chunking, embedding, retrieval, reranking, prompting and generation models, and there is limited guidance on how these components interact.
- We present RAG-NIMBF, a modular benchmarking framework that attaches to existing RAG pipelines as a non-invasive evaluation layer.
- This study validates the framework across multiple generation models on the SQuAD dataset.
- The findings indicate that the retrieval balance of relevance and diversity substantially enhances the relevance and coverage of retrieved context in comparison to a conventional similarity search, with MMR raising nDCG@5 by +4 to +11 percentage points across the swept models.
Sources (1)
- [1]RAG-NIMBF: A Non-Invasive, Modular Benchmarking Framework for Automated RAG Configuration AnalysisarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 08:08 AM
Retrieval-Augmented Generation (RAG) has become the dominant paradigm for grounding the outputs of large language models (LLM) in external knowledge.
However, practitioners are faced with a vast configuration space comprising chunking, embedding, retrieval, reranking, prompting and generation models, and there is limited guidance on how these components interact.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Do Diffusion LLMs Scale? Testing Mercury 3 + Decide
- Oct 19, 2026Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM
- Oct 19, 2026Keep the Lineages Alive: Diversity-Preserving SMC for Diffusion LLM Decoding
- Oct 19, 2026From Black Box to Controller: Inference-Time Steering of LLM via Feature Amplification
- Oct 18, 2026Built a search engine for debugging which is not a wrapper [P]
- Oct 18, 2026CorrectRTL: Boost RTL Generation Quality Through Automated Self-Correction with Instruction Decomposition