Adversarial Benchmarks and BenchDoor Attacks: Exploiting Benchmark Incentives for Backdoor Attacks

Key points

• Machine learning benchmarks act as guideposts for model development and spur the creation of training datasets intended to improve benchmark scores.1
• The adversary either manipulates a benchmark's training or test data, creating an adversarial benchmark, or supplies an independent training dataset while leaving the benchmark unchanged.1
• We formalize five attack scenarios, including training on rewritten test examples and applying similarity-based decontamination.1
• Across multiple-choice Q&A, text classification, and open-ended Q&A with Qwen and Gemma models, BenchDoor achieves positive benchmark gains for all 30 evaluated configurations, with held-out attack success rates of 82.33% to 100.00% at poisoning rates of 2% to 10%.1

Sources (1)

• [1]Adversarial Benchmarks and BenchDoor Attacks: Exploiting Benchmark Incentives for Backdoor Attacks
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 02:49 AM

  We introduce BenchDoor, a family of data poisoning attacks that exploits this incentive.

  Machine learning benchmarks act as guideposts for model development and spur the creation of training datasets intended to improve benchmark scores.

Extractive summary: sentences quoted from the sources.