DecisionRobust: Benchmarking Robustness for JEV Style Decision Models Across Text and Vision

Key points

• Emerging decision models such as JEV provide typed interfaces that map task instructions and observations directly to choices, judgments, and scores.1
• As these models are increasingly used for automated decision-making, understanding their reliability under adversarial manipulation becomes critical.1
• Our benchmark covers practical decision-making scenarios and a diverse suite of attacks, including semantics-preserving text perturbations, image perturbations, and prompt injection in both modalities.1
• We evaluate 30 model configurations on text tasks and 17 on vision tasks, including 24 and 12 held-out models, respectively, that are excluded from candidate selection.1

Sources (2)

• [1]DecisionRobust: Benchmarking Robustness for JEV Style Decision Models Across Text and Vision
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 07:52 PM

  We introduce DecisionRobust, a benchmark for systematically evaluating the robustness of decision models across textual and visual inputs.

  Emerging decision models such as JEV provide typed interfaces that map task instructions and observations directly to choices, judgments, and scores.

• [2]DecisionRobust: Benchmarking Robustness for JEV Style Decision Models Across Text and Vision
  Hugging Face Daily Papers · Oct 13, 07:52 PM · same content

Extractive summary: sentences quoted from the sources.