FlowBERT: An Encrypted Traffic Classification Model Based on Transformers Using Flow Sequence

With the widespread application of network traffic encryption, traffic identification has become increasingly critical. As the types of encryption protocols continue to grow, identifying encrypted traffic with limited training samples has become more challenging. In recent years, pre-training models have been extensively applied in natural language processing due to their ability to utilize a large amount of unlabeled data effectively. However, when applied to encrypted traffic identification, these methods lack sufficient information extraction from encrypted network traffic, resulting in the loss of some essential features and negatively impacting the recognition performance of such approaches. Therefore, we proposed an encrypted traffic classification model based on a Transformer named FlowBERT. In FlowBERT, the semantic features of the traffic can be learned from two dimensions: payload and packet length sequence in large-scale, unlabeled encrypted traffic scenarios. Length sequences are encoded to extract traffic sequence features efficiently, enabling the model to learn the contextual semantic relationships within the sequences. Simultaneously, the pre-training process is improved by balancing data samples, enhancing the performance of the pre-training model. We validated the performance of this method on both classic encrypted traffic classification datasets and the novel network protocol DoH dataset. We concluded that our approach demonstrates robustness and superior recognition performance compared to similar methods.