AI Coding Assistants Show Contradictory Productivity Results
A recent study published on arXiv (2605.01160) uncovers the Productivity-Reliability Paradox (PRP) within AI-enhanced software development. Observations since 2022 reveal a puzzling trend: while controlled experiments indicate productivity increases of 20-56% for clearly defined tasks, a stringent randomized controlled trial (RCT) found a 19% decline in efficiency among seasoned developers. Data from over 10,000 developers indicate a 98% rise in pull requests, yet review times have extended by 91%, with delivery metrics remaining unchanged. The authors attribute this paradox to unpredictable code generators and a lack of specification rigor. Their comprehensive literature review of 67 sources (2022-2026) formally articulates the PRP, introducing three moderating variables and two amplifying mechanisms, along with the AI-Augmented Methodology Taxonomy (AAMT) that categorizes six methodologies into three AI integration frameworks.
Key facts
- Paper arXiv:2605.01160 identifies the Productivity-Reliability Paradox (PRP)
- Controlled studies report 20-56% productivity gains on well-scoped tasks
- Rigorous RCT documents a 19% slowdown for experienced developers
- Telemetry across 10,000+ developers shows 98% more pull requests but 91% longer review times
- PRP arises from non-deterministic code generators and insufficient specification discipline
- Multivocal literature review of 67 sources from 2022-2026
- Three moderating variables: task abstraction, codebase maturity, developer experience
- Two amplifying mechanisms: code review bottleneck, context window constraint
- Proposes AI-Augmented Methodology Taxonomy (AAMT) with six methodologies
Entities
Institutions
- arXiv