Counterfactual Certification for Sequential Unlearning with Long-Run Performance Guarantees

Key points

• For unlearning methods based on fine-tuning on the retain sets, this leads to either degrading performance or increasing computation as the number of unlearning requests grows.1
• For independent successive requests, we instead certify each model release against a counterfactual unlearning trajectory in which the group being deleted never took part.1
• With a fixed fine-tuning budget per deletion, we show that, for smooth strongly convex objectives, this approach keeps the performance loss bounded, uniformly in the number of requests.1
• Experiments with multiclass logistic regression on FEMNIST show that the resulting algorithm needs fewer corrective steps than existing sequential methods and outperforms training-time differential privacy, particularly when group sizes are heterogeneous.1

Sources (1)

• [1]Counterfactual Certification for Sequential Unlearning with Long-Run Performance Guarantees
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 08:56 AM

  Certified machine unlearning ensures the proximity between the unlearned model and a model retrained without the data to be forgotten.

  For unlearning methods based on fine-tuning on the retain sets, this leads to either degrading performance or increasing computation as the number of unlearning requests grows.

Extractive summary: sentences quoted from the sources.