Unknown Attack Detection Method Based on Deep Metric Learning

ABSTRACT Traditional intrusion detection systems (IDS), constrained by the closed‐set classification paradigm, struggle to detect unknown threats in the face of increasingly sophisticated cyberattacks. To address this limitation, we propose an open‐set intrusion detection framework based on deep metric learning, which transforms the unknown attack detection problem into an open‐set recognition problem. The proposed method utilizes a one‐dimensional convolutional neural network (1D CNN) to extract spatiotemporal features from network traffic data, and known centroids are dynamically updated using the feature vector output of the model combined with the iterative weighted average mechanism of the Weiszfeld algorithm. Meanwhile, an improved triplet loss is integrated with cross‐entropy loss to form a hybrid loss function, jointly optimizing model training to enhance intra‐class compactness and inter‐class separability among known attack classes. During the testing phase, the Mahalanobis distance between test samples and class centroids is computed to quantify the statistical divergence between known and unknown classes. The final classification of test samples is determined by comparing this distance against a predefined threshold. Experiments on the CICIDS2017 and UNSW‐NB15 datasets were conducted, and the average accuracy reached 97.28% and 85.47%, respectively. The experimental results show that the method realizes effective detection of unknown attacks while correctly classifying known attacks.