RSI Control: Can AI Models Perform Covert Sabotage in the Self-Improvement Process?
We introduce RSI Control Arena, a testbed in which a frontier model, acting as an autonomous R&D agent, is given a benign main task from a stage of a successor model's training pipeline together with a covert side task to install a misaligned property in the successor.
ProofPaper ↗
Key points
- Recursive self-improvement, in which frontier AI models work on training the next generation, is quickly becoming a reality.
- We study this risk through the lens of AI control.
- We evaluate six frontier models as agents against LLM monitors that read the transcript of the run and the agent's reasoning.
- Our results indicate that recursive self-improvement raises significant safety concerns, and that strong safety effort should be in place before models are trusted with greater responsibility of training their successors.
Sources (1)
- [1]RSI Control: Can AI Models Perform Covert Sabotage in the Self-Improvement Process?arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 06:57 AM
We introduce RSI Control Arena, a testbed in which a frontier model, acting as an autonomous R&D agent, is given a benign main task from a stage of a successor model's training pipeline together with a covert side task to install a misaligned property in the successor.
Recursive self-improvement, in which frontier AI models work on training the next generation, is quickly becoming a reality.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Do Diffusion LLMs Scale? Testing Mercury 3 + Decide
- Oct 20, 2026Cache Engineering for Agents
- Oct 20, 2026AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics
- Oct 19, 2026HarnessPro: Recursive Harness Self-Improvement with Pareto Optimization for Performance-Cost Trade-offs
- Oct 19, 2026Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy
- Oct 18, 2026Trackable Skill Library Lifecycle Management for Agentic Reinforcement Learning