SpecCloak: Probing Harmful Code Generation through Engineering Specifications

Key points

• The code generation capabilities of large language models (LLMs) can equally be misused to implement harmful functionality, underscoring the need to understand their safety behavior in response to programming requests.1
• While request phrasing is known to influence refusal, it remains unclear whether the form in which coding task requirements are stated affects the refusal process for malicious code generation.1
• Building on these findings, we propose SpecCloak, a black-box red-teaming framework combining PseudoSpec with discriminator-guided optimization, achieving an average execution-verified attack success rate of 28.91% across eight target models on RedCode-Gen.1
• We also introduce SpecReveal, a defense prototype combining natural-language intent reconstruction with dual-view detection without target-model fine-tuning.1

Sources (1)

• [1]SpecCloak: Probing Harmful Code Generation through Engineering Specifications
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 07:59 AM

  We introduce PseudoSpec, a template that organizes functional requirements as engineering specifications, and investigate its effects through controlled comparisons, representation analysis, and activation interventions.

  The code generation capabilities of large language models (LLMs) can equally be misused to implement harmful functionality, underscoring the need to understand their safety behavior in response to programming requests.

Extractive summary: sentences quoted from the sources.