FisherEdit: Fisher Information Guided Knowledge Editing for Large Language Models

Key points

• Sequential knowledge editing in Large Language Models (LLMs) often leads to a degradation of general capabilities due to cumulative parameter interference.1
• In this paper, we revisit prominent projection-based methods like AlphaEdit and reveal a surprising mismatch: their robustness is primarily driven by update regularization rather than the intended null-space projection.1
• It penalizes changes to sensitive parameters more strongly while retaining flexibility for new facts, and admits an efficient row-wise closed-form solution.1
• Experiments on Llama-3, Qwen2.5, and Olmo-3 across multiple benchmarks demonstrate that FisherEdit achieves state-of-the-art performance while better preserving model stability and downstream task accuracy.1

Sources (1)

• [1]FisherEdit: Fisher Information Guided Knowledge Editing for Large Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 07:17 AM

  Based on this insight, we propose FisherEdit, a theoretically grounded method built on Fisher geometry, which replaces coarse activation-space projections with importance-aware regularization in parameter space.

  Sequential knowledge editing in Large Language Models (LLMs) often leads to a degradation of general capabilities due to cumulative parameter interference.

Extractive summary: sentences quoted from the sources.