A Two-Stage Framework for Ambiguous Classification in Software Engineering

Classification tasks are prevalent and play a crucial role in the field of software engineering. However, when two classes exhibit similar features at the class level, the classification model is prone to misclassification, which we refer to as ambiguous classification, and the corresponding classes as ambiguous classes. Ambiguous classification may impact the security and reliability of software engineering classification systems.To correct ambiguous classification, we propose a two-stage framework. Our key insight is to combine two different classification models and utilize their complementary knowledge to maximize the classification ability of the two-stage framework. Specifically, we identify ambiguous classes according to the confusion matrix of the original model. Then, we construct a two-stage model, where the first stage utilizes the original model and the second stage utilizes a different model trained on the same dataset. The second-stage model is responsible for reclassifying the samples that are predicted as ambiguous classes by the first-stage model. We evaluate our method on two software engineering tasks. Experimental results indicate that our method can effectively correct ambiguous classification and achieve a relative improvement of 19.8% in F1-score for ambiguous classes.