Effects of Model Scale and Quantization on Social Bias and Abstention in Question Answering

Key points

• We study this problem in Question Answering (QA) using the BBQ benchmark across multiple model sizes and quantization settings from Qwen3, Gemma-3, and SmolLM2.1
• We also examine whether the observed trends remain stable across alternative prompt formulations and when the "Unknown" option is removed in a forced-choice setting.1
• Our results show that larger models achieved higher QA accuracy, but bias showed no simple scaling trend and quantization effects varied across configurations.1
• These findings show that lower observed bias can partly reflect greater abstention, and that fairness evaluation should consider biased responses, abstention, prompt formulation, and answer-option design jointly.1

Sources (1)

• [1]Effects of Model Scale and Quantization on Social Bias and Abstention in Question Answering
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 12:38 PM

  The deployment of small and quantized language models in resource-constrained settings raises an important fairness question: when a compressed model produces fewer biased responses, does this reflect fairer behavior or greater abstention?

  We study this problem in Question Answering (QA) using the BBQ benchmark across multiple model sizes and quantization settings from Qwen3, Gemma-3, and SmolLM2.

Extractive summary: sentences quoted from the sources.