Dynamic Tanh–enhanced transformer architecture for scalable and high-accuracy cyber threat detection in IoT environments

Abstract This study presents a novel intrusion detection approach that uses a Transformer-based architecture enhanced by a Dynamic Tanh (DyT) activation function. To ensure the model concentrates on the most informative inputs while minimizing redundancy, a feature selection stage is carried out using a Random Forest classifier guided by the Minimum Redundancy Maximum Relevance (MRMR) criterion. This step effectively filters out less useful features before training. The DyT activation mechanism allows the model to adjust its threshold values throughout the learning process, enabling faster convergence without sacrificing classification performance. The final model—named Transformer-DyT—was assessed using two widely recognized datasets: IoTID20 and UNSW-NB15, under both binary and multiclass classification settings. The preprocessing phase included normalization, MRMR-RF-based feature selection, addressing class imbalance, and handling invalid values. On IoTID20, the model achieved 97.41% accuracy in nine-class classification, 99.97% in five-class classification, and 99.98% in binary classification, outperforming previous benchmarks. Similarly, it attained 98.26% and 99.99% accuracy in multiclass and binary classification, respectively, on the UNSW-NB15 dataset. These outcomes show that combining a Transformer with DyT activation and MRMR-guided feature selection results in a highly effective detection system. This work directly addresses the challenge of detecting complex cyber threats and modeling temporal patterns in network traffic—an area where many traditional models fall short. By introducing a dynamically adaptive model focused on relevant input features, our approach fills this gap and strengthens intrusion detection in critical IoT and industrial systems.