Training Negatives Still Matter After Calibrating Answerability Readouts

Key points

• Answerability readouts predict whether a passage supplies an answer, but whether calibration can compensate for removing negative passages during training remains unclear.1
• We compare native training, which retains negative passages, with empty training, which removes them, holding questions, labels, positive examples, and model weights fixed.1
• On SQuAD 2.0 and RepLiQA, native-trained logistic readouts at selected depths achieve lower Brier loss for Qwen2.5-7B-Instruct and Gemma-2-9B-it, with or without output scores.1
• Our findings show that training negatives affect ranking and downstream decisions even after common calibration, without establishing a universal advantage for native training.1

Sources (1)

• [1]Training Negatives Still Matter After Calibrating Answerability Readouts
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 02:12 PM

  We present a controlled study of how training-negative construction affects calibrated prediction, ranking, and downstream answer retention.

  Answerability readouts predict whether a passage supplies an answer, but whether calibration can compensate for removing negative passages during training remains unclear.

Extractive summary: sentences quoted from the sources.