FIRM: Fisher-Informed Retention Merging for Federated LLM Unlearning
We propose FIRM (Fisher-Informed Retention Merging), a single-round framework for stable and retention-aware server-side aggregation.
ProofPaper ↗
Key points
- Federated LLM unlearning addresses targeted knowledge removal when forget and retain data remain distributed across private clients and are inaccessible to the server.
- The resulting locally trained adapters encode conflicting objectives, making their aggregation fundamentally different from conventional federated optimization.
- FIRM uses client-local Fisher statistics as a unified sensitivity signal to construct a Fisher-normalized base merge and impose explicit control over retain-sensitive parameters.
- Experiments across WMDP, TOFU, and MUSE-HP using Zephyr-7B and Llama-3-8B show utility gains at comparable forgetting on WMDP and MUSE-HP, with metric-dependent trade-offs on TOFU.
Sources (1)
- [1]FIRM: Fisher-Informed Retention Merging for Federated LLM UnlearningarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 11:39 AM
We propose FIRM (Fisher-Informed Retention Merging), a single-round framework for stable and retention-aware server-side aggregation.
Federated LLM unlearning addresses targeted knowledge removal when forget and retain data remain distributed across private clients and are inaccessible to the server.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 15, 2026FlashAttention with Statistics: Low-Overhead Attention Features for Compact Hallucination Detectors
- Oct 15, 2026Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference
- Oct 15, 2026Training Reasoning Models to Generate Diverse Scientific Ideas
- Oct 15, 2026GlobeBench: Evaluating Text-Based World Model Capabilities for Scalable Environment Simulations
- Oct 14, 2026The Design Space of Tri-Modal Masked Diffusion Models
- Oct 13, 2026Plan or Retrieve First? A Scale Study of Tool Discovery Architectures