Evidence Sufficiency and Model Comparisons in MexRegQA: An Item-Level Audit of Mexican-Spanish Regulatory QA

Key points

• We audit Mexican-Spanish financial-regulatory QA and find that 35 of 135 covered items lack some or all load-bearing facts.1
• Removing evidence-defective items changes the nominal significance status of 14 of 55 pairwise comparisons.1
• Building on sufficient-context research, we contribute MexRegQA v1.0: 192 items with provenance and evidence-status labels, frozen retrieval evidence for 191 evaluated items, predictions, judge outputs and offline checks.1
• We propose releasing evidence-status labels alongside grounded QA benchmarks so that model comparisons can be examined separately from evidence availability.1

Sources (1)

• [1]Evidence Sufficiency and Model Comparisons in MexRegQA: An Item-Level Audit of Mexican-Spanish Regulatory QA
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 02:54 AM

  Similarity-gated coverage does not establish that retrieved evidence contains the facts needed to answer a question.

  We audit Mexican-Spanish financial-regulatory QA and find that 35 of 135 covered items lack some or all load-bearing facts.

Extractive summary: sentences quoted from the sources.