Claude Boosts Codex Code Review Accuracy by 18 Percentage Points
A recent investigation published on arXiv (2607.21656) explores the effectiveness of pairing two coding agents—one tasked with writing code and the other with reviewing it. The researchers conducted a controlled study involving 116 recent hard and medium LCb tasks, utilizing Claude and Codex under six different conditions: solo baselines, same-model orderings, and cross-model orderings. In this setup, the reviewer could see the problem and the writer's draft but was unable to run tests, mimicking a code review scenario. Claude’s review of Codex drafts improved the pass rate from 71.6% to 89.7% (p=0.001), whereas Codex’s self-review increased it to 84.5% (p=0.022). Conversely, Codex reviewing Claude drafts resulted in a decline from 91.4% to 82.8% (p=0.046), with Claude’s self-review maintaining the 91.4% baseline. These results indicate that cross-model pairing is advantageous only when the superior model (Claude) evaluates the inferior one (Codex), offering valuable insights for developers employing multiple LLMs in code review processes.
Key facts
- Study tested Claude and Codex on 116 LCb tasks
- Claude review of Codex drafts raised pass rate from 71.6% to 89.7%
- Codex self-review raised pass rate from 71.6% to 84.5%
- Codex review of Claude drafts dropped pass rate from 91.4% to 82.8%
- Claude self-review left baseline unchanged at 91.4%
- Reviewer could not execute tests, simulating code review
- Cross-model pairing beneficial only when stronger model reviews weaker one
- Study published on arXiv with ID 2607.21656
Entities
Institutions
- arXiv