Probe-Geometry Alignment Erases LLM Memorization Below Chance

A new study on arXiv (2605.01699) demonstrates that adversarial probes can detect memorization traces in large language models even after behavioral unlearning, but these traces can be surgically removed without capability loss. The protocol uses a leave-one-out cross-sequence probe to test generalization of memorization signatures across held-out sequences. Signatures were consistent across scales: memorization-specific gaps of +0.32, +0.19, and +0.30 on Pythia-70M, GPT-2 medium, and Mistral-7B respectively. On Pythia-70M, random-initialization control collapsed to -0.04 at the deepest layer where pretrained signature peaked. The probe direction is causally separable from recall: projecting it out collapses the signature locally (+0.44 to -0.19) while behavioral recall barely changes. A probe trained on naturally memorized content does not classify fine-tuning-injected secrets, indicating two distinct representations.