SecCodePLT: A Unified Platform for Evaluating the Security of Code GenAI
Yu Yang, Yuzhou Nie, Zhun Wang, Yuheng Tang, Wenbo Guo, Bo Li, Dawn Song
2024-10-16
Summary
This paper introduces SecCodePLT, a new platform designed to evaluate the security risks associated with code generation AI models, focusing on their ability to produce insecure code and assist in cyberattacks.
What's the problem?
Existing benchmarks for assessing the security of code generation AI have limitations. They often focus too much on whether a model can suggest attacks rather than actually generating executable ones. Additionally, many benchmarks use static evaluation methods that may not accurately reflect real-world performance, and expert-verified benchmarks are usually limited in scale.
What's the solution?
To overcome these issues, the authors developed SecCodePLT, which provides a unified and comprehensive evaluation of code generation AIs. They created a new method for generating high-quality data that combines expert input with automated techniques. This allows for large-scale testing while ensuring data quality. The platform also includes dynamic evaluations that simulate real-world scenarios to better assess the models' abilities to generate insecure code and assist in cyberattacks.
Why it matters?
This research is important because it addresses critical gaps in evaluating the security of AI models that generate code. By providing a more accurate and comprehensive assessment tool, SecCodePLT can help developers identify and mitigate potential security risks in their AI systems, ultimately leading to safer software development practices.
Abstract
Existing works have established multiple benchmarks to highlight the security risks associated with Code GenAI. These risks are primarily reflected in two areas: a model potential to generate insecure code (insecure coding) and its utility in cyberattacks (cyberattack helpfulness). While these benchmarks have made significant strides, there remain opportunities for further improvement. For instance, many current benchmarks tend to focus more on a model ability to provide attack suggestions rather than its capacity to generate executable attacks. Additionally, most benchmarks rely heavily on static evaluation metrics, which may not be as precise as dynamic metrics such as passing test cases. Conversely, expert-verified benchmarks, while offering high-quality data, often operate at a smaller scale. To address these gaps, we develop SecCodePLT, a unified and comprehensive evaluation platform for code GenAIs' risks. For insecure code, we introduce a new methodology for data creation that combines experts with automatic generation. Our methodology ensures the data quality while enabling large-scale generation. We also associate samples with test cases to conduct code-related dynamic evaluation. For cyberattack helpfulness, we set up a real environment and construct samples to prompt a model to generate actual attacks, along with dynamic metrics in our environment. We conduct extensive experiments and show that SecCodePLT outperforms the state-of-the-art (SOTA) benchmark CyberSecEval in security relevance. Furthermore, it better identifies the security risks of SOTA models in insecure coding and cyberattack helpfulness. Finally, we apply SecCodePLT to the SOTA code agent, Cursor, and, for the first time, identify non-trivial security risks in this advanced coding agent.