CursorCore: Assist Programming through Aligning Anything
Hao Jiang, Qi Liu, Rui Li, Shengyu Ye, Shijin Wang
2024-10-10
Summary
This paper presents CursorCore, a new framework designed to improve programming assistance by effectively integrating various types of information during the coding process.
What's the problem?
While large language models (LLMs) can help with programming tasks like code completion and editing, they often struggle to combine different sources of information, such as past coding history, current code, and user instructions. This lack of integration can make their assistance less effective and more manual than it should be.
What's the solution?
To address this issue, the authors developed CursorCore, which includes a new evaluation benchmark called APEval to assess how well models align with different types of programming information. They also created a data generation pipeline called Programming-Instruct that automatically generates diverse training data from sources like GitHub. By generating 219,000 samples and fine-tuning multiple models, CursorCore improves the performance of programming assistants by unifying different applications like inline chat and automated editing.
Why it matters?
This research is significant because it enhances how AI can assist programmers by making the interaction more seamless and efficient. By improving the way models understand and utilize various information sources, CursorCore can lead to better coding assistants that help users write code faster and with fewer errors, ultimately benefiting software development as a whole.
Abstract
Large language models have been successfully applied to programming assistance tasks, such as code completion, code insertion, and instructional code editing. However, these applications remain insufficiently automated and struggle to effectively integrate various types of information during the programming process, including coding history, current code, and user instructions. In this work, we propose a new conversational framework that comprehensively integrates these information sources, collect data to train our models and evaluate their performance. Firstly, to thoroughly evaluate how well models align with different types of information and the quality of their outputs, we introduce a new benchmark, APEval (Assist Programming Eval), to comprehensively assess the performance of models in programming assistance tasks. Then, for data collection, we develop a data generation pipeline, Programming-Instruct, which synthesizes training data from diverse sources, such as GitHub and online judge platforms. This pipeline can automatically generate various types of messages throughout the programming process. Finally, using this pipeline, we generate 219K samples, fine-tune multiple models, and develop the CursorCore series. We show that CursorCore outperforms other models of comparable size. This framework unifies applications such as inline chat and automated editing, contributes to the advancement of coding assistants. Code, models and data are freely available at https://github.com/TechxGenus/CursorCore.