HCRP‐OSD: Fine‐Grained Open‐Set Intrusion Detection Based on Hybrid Convolution and Adversarial Reciprocal Points Learning

ABSTRACT Amidst the swift progression of network technology, unknown network attacks and malicious code iterations perpetually surface, thereby imposing augmented exigencies on the efficacy and innovativeness of intrusion detection systems within networks. Timely detection of unknown network attacks is critical to reducing the risk of significant damage to systems. This paper aims to develop an open‐set intrusion detection model that is able to infer unknown network attacks and correctly classify known attacks. In order to enable the model to classify known attacks while inferring unknown attacks correctly, we consider the open space risk of unknown attacks when training the known attacks classification model. Specifically, we propose an open‐set intrusion detection system, HCRP‐OSD, consisting of three modules: Network flow feature extraction, hybrid convolutional network, and ARPL open‐set intrusion detection. The network flow feature extraction module extracts data information from the original network traffic to avoid the loss of original information caused by manual design and selection of features. The hybrid convolutional network module learns distinguishable features between different known attacks. The hybrid convolutional network uses two learning channels, a two‐dimensional CNN and a one‐dimensional residual network, to obtain attack features from different angles. The features are aggregated at the output layer. The ARPL open‐set intrusion detection module learns a set of vectors as reciprocal points for each known attack and maximizes the distance between the known attack and its reciprocal point during training. This increases the discrimination between known and unknown attacks while accurately classifying known attacks. Experiments on the dataset CICIDS2017 show that our method outperforms the baseline models. The AUROC for identifying unknown attacks is 97.59%. The classification accuracy for known attacks is 99.97%.