Refusal Is Not Robustness: Measuring Harmful Reachability in Autoregressive Language Models

Key points

• Autoregressive language models define distributions over possible outputs, and a sampled refusal can coexist with substantial probability mass on harmful generation paths.1
• We study this hidden structure through bounded continuation analysis (BCA), which models autoregressive generation as a stochastic process over token prefixes and computes the finite-horizon probability that generation reaches a harmful continuation.1
• BCA constructs a bounded continuation tree, labels terminal continuations with a semantic harm evaluator, and estimates harmful reachability using exact probabilistic model checking.1
• Across four instruction-tuned LLMs and 100 JailbreakBench behaviors, we show that BCA effectively quantifies hidden harmful behavior and provides a useful signal to guide red teaming.1

Sources (1)

• [1]Refusal Is Not Robustness: Measuring Harmful Reachability in Autoregressive Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 11:45 PM

  LLM safety evaluations via jailbreaking typically index on attack success rate (ASR): whether a generated response violates safeguards.

  Autoregressive language models define distributions over possible outputs, and a sampled refusal can coexist with substantial probability mass on harmful generation paths.

Extractive summary: sentences quoted from the sources.