VulCoBERT: A CodeBERT-Based System for Source Code Vulnerability Detection

As we advance in time, with the evolution of the computer industry and the escalating intricacy of diverse software, the demand for source code defects is also increasing. Traditional deep learning-based software defect detection methods perform well on synthetic defect datasets, but their performance on real software defect datasets is unsatisfactory. At the same time, pre-trained models derived from extensive data training are extensively employed in various NLP tasks and have achieved excellent results. Based on this background, this study introduces a software defect detection system leveraging CodeBERT and Bi-LSTM. This method first preprocesses and standardizes the C source code to minimize the impact of redundant information and reduce noise; Secondly, coding sequences are segmented and encoded using the CodeBERT pre-trained model, capturing the semantic features within the program's code and converting them into vectors containing code feature information and output; Then, the output of the received CodeBERT is processed through the Bi-LSTM network to acquire the structural composition inherent to the code and the semantic information of the positive and negative terms; Finally, the vectors containing source code features are classified using a fully connected network to determine whether the code segment has defects. To verify the impact of this approach, we used the cross-language benchmark test set CodeXGLUE proposed by Microsoft Research Institute for evaluating code tasks for validation. The results showed that this method had higher accuracy in detecting real software defect samples than other methods, indicating that the proposed method can effectively improve software defect detection capabilities.