Token-Level Differential Privacy for Continual Learning

A new framework, Privacy-enhanced Continual Learning (PeCL), proposes token-level dynamic differential privacy to protect sensitive data in continual learning models. Traditional uniform DP budgets degrade model utility by indiscriminately protecting all data. PeCL adaptively allocates privacy budgets based on semantic sensitivity of individual tokens, ensuring robust protection for private entities while minimizing noise for non-sensitive knowledge. It also integrates a privacy-guided memory sculpting module that leverages sensitivity analysis to selectively forget sensitive information and retain important general knowledge. This approach addresses privacy challenges in CL deployment for sensitive areas like healthcare and finance.