PROBE: Learning to Audit Policy Compliance in Tool-Using LLM Agents
We propose PROBE (Policy Rule Observation via Behavioral Exploration), an algorithm for training LLM-based auditors that interact with deployed agents as realistic users and probe them for policy violations through multi-turn, tool-mediated dialogue.
ProofPaper ↗
Key points
- Modern agentic systems extend language models with external tools, enabling them to complete complex tasks through multi-turn interactions.
- We formalize agent auditing as a two-player Markov game and propose a composite reward that balances violation discovery, behavioral plausibility, conversation progress, and policy diversity.
- Our results show that multi-turn behavioral auditing is a distinct and necessary axis of agent evaluation, complementing capability and safety benchmarks.
- PROBE offers a scalable path toward automated policy auditing for real-world agentic systems and suggests a co-evolutionary loop in which auditors expose failures and agents improve from auditor feedback.
Sources (1)
- [1]PROBE: Learning to Audit Policy Compliance in Tool-Using LLM AgentsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 04:01 PM
We propose PROBE (Policy Rule Observation via Behavioral Exploration), an algorithm for training LLM-based auditors that interact with deployed agents as realistic users and probe them for policy violations through multi-turn, tool-mediated dialogue.
Modern agentic systems extend language models with external tools, enabling them to complete complex tasks through multi-turn interactions.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026Rethinking the Role of Tool Graph Memory in Multi-Turn Tool Use
- Oct 15, 2026Incentivizing Tool-Use Reinforcement Learning for Post-Arrival Emergency Care Prediction
- Oct 15, 2026Each Reward to Its Relevant Tokens: Reward-Specific Token-Level Credit Assignment for Multi-Reward RL
- Oct 15, 2026Bellman-Consistent Trajectory RL for LLM Arithmetic Reasoning
- Oct 14, 2026Gibbs-Gap Policy Optimization for Off-Policy RL with Verifiable Rewards
- Oct 14, 2026From Failures to Learnable Worlds: Policy-Conditioned Environment Generation for Navigation Agents
