Behavioral Forgetting Is Not Privacy in Machine Unlearning

Key points

• We argue that such behavioral forgetting is an insufficient standard for privacy-oriented unlearning, and that evaluation should instead test whether forgetting persists across the access paths through which the same information may be recovered.1
• We demonstrate this with a controlled case study of cross-lingual querying: across multiple multilingual model families, forgetting varies substantially when semantically equivalent information is queried in different languages and scripts.1
• A single access path is therefore enough to expose a real gap between behavioral forgetting and information inaccessibility, motivating broader evaluation of access-path generalization and adversarial recoverability alongside in-distribution forgetting.1

Sources (1)

• [1]Behavioral Forgetting Is Not Privacy in Machine Unlearning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 12:48 PM

  Machine unlearning aims to remove targeted information from large language models, but existing evaluations often measure forgetting only on a designated benchmark distribution.

  We argue that such behavioral forgetting is an insufficient standard for privacy-oriented unlearning, and that evaluation should instead test whether forgetting persists across the access paths through which the same information may be recovered.

Extractive summary: sentences quoted from the sources.