CyberForge: Framework for Generating Repository-Level Security Training Data

A novel framework named CyberForge has been launched to tackle the shortage of security training data for large language model (LLM) agents. This framework creates executable, repository-level security training data by incorporating vulnerabilities into actual C/C++ projects. Each instance undergoes dynamic validation: the modified build must successfully pass the project's unit tests, and a generated proof-of-vulnerability (PoV) should activate on the altered build but remain inactive on the unaltered version. This method is not constrained by the number of disclosed vulnerabilities, offering a scalable means for training defensive agents. The details are outlined in a paper on arXiv (arXiv:2608.06471), categorized as 'cross'. The paper discusses the difficulties faced by frontier LLM agents in identifying and fixing intricate vulnerabilities in real software, emphasizing that while readily available agents can assist attackers, defenders must persistently locate and rectify all vulnerabilities within rapidly expanding codebases. CyberForge seeks to enhance defensive agents by supplying high-quality training data along with reproducible build and execution environments.