Log Anomaly Detection method based on BERT model optimization

In the field of computer system anomaly detection, log anomaly detection is a very important project. In order to detect system faults from log text data accurately and quickly, this paper proposes a log anomaly detection method, namely Prog-BERT-LSTM, which uses the network based on the BERT model as the text vectorization module, and designs the sequence feature learning module based on LSTM to avoid the loss of sequence features caused by the disappearance of gradient in the calculation process, and further obtain the semantics and features of the input log sequence text. The progressive masking strategy is used to aggregate the text semantic vector and sequence feature vector. We compare the Prog-BERT-LSTM model with the BERT-based model (LogBERT) on three public log datasets. The test results show that the Prog-BERT-LSTM model has better performance than the standard BERT-based model (LogBERT).