Deep learning or classical machine learning? An empirical study on line‐level software defect prediction

Abstract Background Line‐level software defect prediction (LL‐SDP) serves as a valuable tool for developers to detect defective lines with minimal human effort. Recently, GLANCE was proposed as a readily implementable baseline for assessing the efficacy of newly proposed LL‐SDP models. Problem While DeepLineDP, a cutting‐edge LL‐SDP model rooted in deep learning, has demonstrated state‐of‐the‐art performance, it has not yet been compared against GLANCE. Objective We aim to empirically compare DeepLineDP with GLANCE to obtain a comprehensive understanding of how deep learning contributes to solving the LL‐SDP challenge. Method We compare GLANCE against DeepLineDP to assess the extent to which DeepLineDP surpasses GLANCE in predicting defective files and identifying problematic lines. In order to obtain a reliable conclusion, we use the same dataset and performance metrics utilized by DeepLineDP. Result Our experimental findings indicate that DeepLineDP does not outperform GLANCE in LL‐SDP. This suggests that the application of deep learning, in this context, does not yield the anticipated significant improvements. Conclusion This finding underscores the need for further research in deep learning‐based LL‐SDP to attain the state‐of‐the‐art performance that remains elusive for less advanced techniques.