Enhancing Aspect Category Detection Through Hybridised Contextualised Neural Language Models: A Case Study In Multi-Label Text Classification

Abstract Recently, the field of Natural Language Processing (NLP) has made significant progress with the evolution of Contextualised Neural Language Models (CNLMs) and the emergence of large LMs. Traditional and static language models exhibit limitations in tasks demanding contextual comprehension due to their reliance on fixed representations. CNLMs such as BERT and Semantic Folding aim to produce feature-rich representations by considering a broader linguistic context. In this paper, Deep Learning-based Aspect Category Detection approaches are introduced to perform text classification. The study extensively assesses classification model performance, emphasising enhanced representativeness and optimised feature extraction resolution using CNLMs and their hybridised variants. The effectiveness of the proposed approaches is evaluated on benchmark datasets of 4500 reviews from the laptop and restaurant domains. The results show that the proposed approaches using hybridised CNLMs outperform state-of-the-art methods with an f-score of 0.85 for the laptop and f-scores higher than 0.90 for the restaurant dataset. This study represents a pioneering work as one of the initial research efforts aiming to jointly evaluate the representation performance of CNLMs with different architectures to determine their classification capabilities. The findings indicate that the proposed approaches can enable the development of more effective classification models in various NLP tasks.