PRISM: Transfer and Limits of Activation Interventions in Tool-Calling Language Models

Key points

• Recent literature shows that unsafe tool calls can often accompany textual refusals in language models.1
• Our findings demonstrate that leveraging lower-resource settings (specifically quantized NF4 labels and cross-lingual transfer) offers a robust pathway to expose and improve model alignment.1
• Furthermore, these alignment improvements transfer effectively across languages: five-language NF4-supervised directions successfully transfer to held-out Arabic, reducing unsafe violations in Llama-3.1-8B by 21.95 percentage points without target-language tuning.1
• Ultimately, PRISM highlights how auxiliary supervision from compressed models and multilingual data can uncover and mitigate hazardous tool-calling behaviors across different model families.1

Sources (1)

• [1]PRISM: Transfer and Limits of Activation Interventions in Tool-Calling Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 06:55 PM

  To investigate this hidden vulnerability, we introduce PRISM, a framework for evaluating the transfer of activation interventions across precision, languages, and scales.

  Recent literature shows that unsafe tool calls can often accompany textual refusals in language models.

Extractive summary: sentences quoted from the sources.