A lightweight IoT intrusion detection model based on improved BERT-of-Theseus

The proliferation of Internet of Things (IoT) technology has resulted in an increase in security vulnerabilities associated with the interconnectivity of IoT devices. As a result, there is a need for intrusion detection mechanisms that can effectively detect attacks on IoT security vulnerabilities. However, due to the resource constraints of IoT deployment devices, intrusion detection schemes must be customized to meet the specific demands of the IoT environment. In this study, we propose a knowledge-distillation-based IoT intrusion detection model named BT-TPF, which is capable of detecting network attacks encountered by IoT devices in an IoT environment with limited computing resources. The proposed BT-TPF model leverages a Siamese network for feature dimensionality reduction of complex high-dimensional network traffic data. Additionally, it employs a large-scale Vision Transformer as a teacher model to guide a small-scale Poolformer model during training, before deploying the trained Poolformer model as a classifier to detect network intrusion traffic. Through knowledge distillation, the final small model obtained in this paper only requires a minimum of 788 parameters, reducing the number of parameters by approximately 90% compared to the large model before knowledge distillation, while maintaining high detection accuracy. Experimental results show that the BT-TPF model achieves over 99% accuracy on both the CIC-IDS2017 and TON_IoT datasets. Furthermore, it exhibits significant advantages compared to traditional Deep Learning methods and recent state-of-the-art models, as evidenced by various evaluation metrics.