LogAttn: Unsupervised Log Anomaly Detection with an AutoEncoder Based Attention Mechanism

System logs produced by modern computer systems are valuable resources for detecting anomalies, debugging performance issues, and recovering application failures. With the increasing scale and complexity of the log data, manual log inspection is infeasible and man-power expensive. In this paper, we proposed LogAttn, an autoencoder model that combines an encoder-decoder structure with an attention mechanism for unsupervised log anomaly detection. The unstructured normal log data is proceeded by a log parser that uses a semantic analyse and clustering algorithm to parse log data into a sequence of event count vectors and semantic vectors. The encoder combines deep neural networks with an attention mechanism that learns the weights of different features to form a latent feature representation, which is further used by a decoder to reconstruct the log event sequence. If the reconstruction error is above a predefined threshold, it detects an anomaly in the log sequence and reports the result to the administrator. We conduct extensive experiments based on three real-world log datasets, which show that LogAttn achieves the best comprehensive performance compared to the state-of-the-art methods.