ADA-NAF: Semi-Supervised Anomaly Detection Based on the Neural Attention Forest

In this study, we present a novel model called ADA-NAF (Anomaly Detection Autoencoder with the Neural Attention Forest) for semi-supervised anomaly detection that uniquely integrates the Neural Attention Forest (NAF) architecture which has been developed to combine a random forest classifier with a neural network computing attention weights to aggregate decision tree predictions. The key idea behind ADA-NAF is the incorporation of NAF into an autoencoder structure, where it implements functions of a compressor as well as a reconstructor of input vectors. Our approach introduces several technical advances. First, a proposed end-to-end training methodology over normal data minimizes the reconstruction errors while learning and optimizing neural attention weights to focus on hidden features. Second, a novel encoding mechanism leverages NAF’s hierarchical structure to capture complex data patterns. Third, an adaptive anomaly scoring framework combines the reconstruction errors with the attention-based feature importance. Through extensive experimentation across diverse datasets, ADA-NAF demonstrates superior performance compared to state-of-the-art methods. The model shows particular strength in handling high-dimensional data and capturing subtle anomalies that traditional methods often do not detect. Our results validate the ADA-NAF’s effectiveness and versatility as a robust solution for real-world anomaly detection challenges with promising applications in cybersecurity, industrial monitoring, and healthcare diagnostics. This work advances the field by introducing a novel architecture that combines the interpretability of attention mechanisms with the powerful feature learning capabilities of autoencoders.