Similar Bug Reports Recommendation System using BERT

In order to document software issues so that they can be later analyzed and corrected, Bug Reports (BR) are used. According to Mozilla’s Bugzilla, as an example, over 8,000 new bugs were reported for Firefox in 2020. Thus, a recommendation system can be a valuable tool to improve productivity in software development, especially when dealing with a high volume of BRs to be reviewed and possibly fixed by the maintainers. This study proposes and evaluates a BR recommendation system based on textual similarity, with the differential use of the state-of-the-art text comprehension model BERT as one of the factors in the similarity calculation. We use a dataset with 106k Mozilla BRs extracted from Bugzilla, an open-source platform. The main objective is to improve suggestions for BRs with a context close to that provided by the maintainer. In the study, we experimented with the BERT model adopting the similarity calculation as individually as together with the well-known TF-IDF vectorization technique. The results attest that there were gains of approximately 14% in the frequency of relevant BRs for the first 20 recommendations compared to a baseline technique that adopts only the TF-IDF vectorization approach. The BERT model added improvements to the evaluated metrics (precision, feedback, and likelihood) when complementary to TF-IDF, but did not perform positively in an isolated manner. Overall, the findings could have implications for software development teams handling a high volume of BRs and potentially increase their productivity in resolving BRs.