RA$^2$: Retain-Anchored Attraction Preserves Forget-Adjacent Utility in LLM Unlearning
We propose RA$^2$ (Retain-Anchored Attraction), a latent-space unlearning method that redirects forget-token states toward nearby retain-supported representations while preserving higher-layer behavior on retain data.
ProofPaper ↗
Key points
- Large Language Model (LLM) unlearning is often evaluated primarily by target suppression and clean retain utility.
- A fundamental question is: where should forget representations go after they leave the removed behavior?
- Successful unlearning depends not only on whether forget-conditioned representations depart from the harmful computation, but also on the direction in which they are redirected afterward.
- On WMDP and MUSE, RA$^2$ yields higher forget-adjacent utility than strong unlearning baselines at similar broad-utility levels.
Sources (1)
- [1]RA$^2$: Retain-Anchored Attraction Preserves Forget-Adjacent Utility in LLM UnlearningarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 01:31 AM
We propose RA$^2$ (Retain-Anchored Attraction), a latent-space unlearning method that redirects forget-token states toward nearby retain-supported representations while preserving higher-layer behavior on retain data.
Large Language Model (LLM) unlearning is often evaluated primarily by target suppression and clean retain utility.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 14, 2026ReaDy: Recurrent States as Dynamic Context for Speculative Drafting in Hybrid Linear Attention Large Language Models
- Oct 13, 2026Gemini 4 Argon: Built to solve your hardest problems
- Oct 13, 2026Can Agents Forget When Asked? A Benchmark for Memory Unlearning in LLM Agents
- Oct 13, 2026Hybrid Reasoning Agents
- Oct 13, 2026DeltaTTG: Training Multi-Turn Agents with a Token-to-Go Progress Meter
- Oct 12, 2026[AINews] Meta Muse Wonder & Wonder Full: MSL’s Claude 5.5-tier Frontier Model