SELFCI: A Self-Distillation Framework for Privacy in LLMs

A new framework called SELFCI (Self-Distillation for Contextual Integrity) aims to improve privacy in large language models by decoupling information suppression from task resolution. Proposed in a paper on arXiv (2605.20258), SELFCI uses complementary self-distillation to optimize two independent reverse KL divergences: one preserves task-relevant information for utility, the other enforces minimal disclosure. This creates a Product-of-Experts target that balances privacy and performance without degrading task accuracy. The approach addresses Contextual Integrity (CI), which governs information flows according to contextual norms, a critical issue as LLMs are deployed as personal agents handling sensitive workflows.