DECODE Dataset Tracks 53.6K Real-World Edits of AI-Generated Code
A new dataset named DECODE (Developer Edits of Code Dataset) has been launched, encompassing 53.6K authentic in-IDE code modifications of AI-generated code in Python, TypeScript, and JavaScript, collected from more than 1,000 developers. This dataset aims to fill the gap in high-quality, realistic code editing data necessary for training large language models, which typically depend on Git commits that only reflect final, successful code. Findings indicate that the majority of edits happen within the first 15 minutes of accepting an AI suggestion, with 31% of editing paths resulting in the complete removal of AI-generated completions. The research sheds light on the timing, reasoning, and methods behind developers' alterations to AI-generated code.
Key facts
- DECODE dataset contains 53.6K real-world code edits
- Edits are from AI-generated code in Python, TypeScript, and JavaScript
- Data sourced from over 1,000 developers
- Most edits occur within 15 minutes of accepting AI completion
- 31% of edit trajectories involve removal of AI completions
- Dataset addresses gap in training data for LLMs
- Git commits only contain final successful code snippets
- Study provides insights on editing behavior
Entities
—