Spatial-temporal knowledge distillation for lightweight network traffic anomaly detection

Deep learning-based network traffic anomaly detection methods have been the mainstream approaches to enhancing the accuracy performance of Network Intrusion Detection Systems (NIDSs). However, there are several problems that remain to be addressed in practical scenarios. First, the memory and computing power of intelligent terminals restrict the deployment of computationally intensive deep learning methods. Second, the depth and width of representations are of central significance for the accuracy of detection, at the cost of memory consumption and computational complexity. Third, the long tail effect spawned by the category imbalance of network traffic is prevalent in real-world fine-grained anomaly detection tasks. Therefore, we propose a Spatial-Temporal Knowledge Distillation (STKD) algorithm framework for lightweight network traffic anomaly detection to tackle the challenges. Integrating multi-scale One-Dimensional Convolutional Neural Network (1D CNN) and Long Short-Term Memory Network (LSTM), and adopting identity mapping, we propose a Multi-Scale Spatial-Temporal Residual Network (MSSTRNet) as the teacher model for deep spatial-temporal feature extraction of network traffic. Based on Knowledge Distillation (KD), we compress MSSTRNet to the lightweight student model named LENet which is suitable for deployment. Introducing Focal Loss (FL) instead of Cross Entropy (CE) Loss into the KD process, we attempt to alleviate the long tail effect in the fine-grained anomaly detection tasks. Experiments demonstrate the superiority of our proposed methods on accuracy performance, memory consumption and computation complexity.