Adversarial Test-Hardening Loop for AI-Written Code: Instrument Artifact Uncovered

A recent investigation published on arXiv (2607.23002) explores an adversarial test-hardening loop for code produced by large language models. This loop involves a Tester model that generates tests, followed by mutation testing to identify surviving injected defects, and a Critic model that creates tests aimed at eliminating those defects, with all outcomes determined mechanically. In Experiment 1, conducted on five Python subjects, the loop successfully eliminated 105 mutants that the one-shot generation overlooked, with no losses. Notably, an autopsy revealed that a previously noted cross-lineage effect (p = 9.5e-66) was actually an artifact from an output cap that truncated the verbose model, a detail uncovered only through adversarial review of the analysis. Additional confounding was present in each study arm.