Deep Q-Network-Based Open-Set Intrusion Detection Solution for Industrial Internet of Things

Industrial Internet of Things (IIoT) has brought a lot of convenience for the industrial world to digitization, automation and intelligence, but it inevitably introduces inherent cyber security risks, resulting in an issue that traditional intrusion detection techniques are no longer sufficient for IIoT environments. To solve this issue, we propose an open-set solution called DC-IDS for IIoT based on deep reinforcement learning. In this solution, the open-set recognition problem in intrusion detection is modeled as a discrete-time Markov decision process, and Deep Q-Network (DQN) is employed to solve it. Meanwhile, a Conditional Variational Auto-Encoder is introduced to the value network in DQN. Therefore, the open-set recognition problem in intrusion detection is divided into two subproblems, namely known traffic fine-grained classification problem and unknown attacks recognition problem. We use DQN to solve the known traffic fine-grained classification problem. Since the reconstruction error of known traffic is generally smaller than the reconstruction error of unknown attacks, we use reconstruction error to recognize unknown attacks. Experiments on IIoT dataset TON-IoT demonstrate the effectiveness of DC-IDS model, which achieves better performance in terms of the recognition rate of unknown attacks as well as the stability of the model compared to previous proposed methods.