BanglaBBQ: A Bias Benchmark for Question Answering in the Bangladeshi Context

Key points

• Following KoBBQ and PakBBQ, BanglaBBQ adapts the original Bias Benchmark for Question Answering (BBQ) by classifying its templates into three groups: directly translated (DT), modified to align target groups or contexts with Bangladesh (TM), and removed when deemed unsuitable for the Bangladeshi context (SR).1
• Five newly authored (NA) categories, based on stereotypes common in Bangladeshi society, are then added: marital status, age and political perception, regional origin, educational background, and mental-health status.1
• Each evaluated template also has a counter-stereotypical version of its disambiguated context.1
• The dataset was evaluated using two open-weight models (Qwen3-4B and Qwen3-8B) and two closed models (GPT-4.1-mini and Gemini 2.5 Flash).1

Sources (1)

• [1]BanglaBBQ: A Bias Benchmark for Question Answering in the Bangladeshi Context
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 04:12 PM

  This paper introduces BanglaBBQ, a question-answering bias benchmark designed to capture stereotypes grounded in the Bangladeshi context, in both Bangla and English.

  Following KoBBQ and PakBBQ, BanglaBBQ adapts the original Bias Benchmark for Question Answering (BBQ) by classifying its templates into three groups: directly translated (DT), modified to align target groups or contexts with Bangladesh (TM), and removed when deemed unsuitable for the Bangladeshi context (SR).

Extractive summary: sentences quoted from the sources.