A Unified PAC-Bayesian Framework for Machine Unlearning via Information Risk Minimization
A Unified PAC-Bayesian Framework for Machine Unlearning via Information Risk Minimization
Sharu Theresa Jose,O. Simeone
TLDR
A unified PAC-Bayesian framework for machine unlearning is developed that recovers the two recent design principles - variational unlearning and forgetting Lagrangian as information risk minimization problems and can be interpreted as PAC- Bayesian upper bounds on the test loss of the unlearned model.
Abstract
Machine unlearning refers to mechanisms that can remove the influence of a subset of training data upon request from a trained model without incurring the cost of re-training from scratch. This paper develops a unified PAC-Bayesian framework for machine unlearning that recovers the two recent design principles - variational unlearning [1] and forgetting Lagrangian [2] as information risk minimization problems [3]. Accordingly, both criteria can be interpreted as PAC-Bayesian upper bounds on the test loss of the unlearned model that take the form of free energy metrics.