Ultra fast encrypted network traffic classification based on classification by retrieval powered by out-of-distribution

The dominance of encryption and the relentless growth of network traffic have rendered traditional deep packet inspection ineffective and made retraining-based machine learning classifiers operationally costly, as every new application or threat class requires a full model rebuild. In order to overcome this challenge, we present an ultra-fast encrypted traffic classification system based on Classification By Retrieval (CBR): each flow is represented by a fixed-length feature vector extracted from only the first few packets, indexed in a HNSW approximate nearest-neighbor structure, and classified by retrieving its nearest labeled neighbor. New classes are absorbed by inserting a handful of labeled vectors into the running index, with no retraining and no downtime, and out-of-distribution traffic is detected within the same retrieval step using a single global distance threshold. We evaluate handcrafted statistical features and learned CNN and MLP embeddings against a Random Forest baseline on the public CESNET-TLS22 dataset (184 application classes) and a proprietary cellular dataset (RADCOM). With CNN embeddings, CBR matches Random Forest accuracy (macro F1 of 0.923 versus 0.925 on CESNET) while training roughly ten times faster, adapts to new classes two to three orders of magnitude faster (seconds versus thousands of seconds), and detects out-of-distribution flows at up to 0.92 AUROC. The retrieval index is also far cheaper to keep resident than the baseline it replaces, holding 1.2 million CESNET flows in 171 MB against 1,350 MB for the serialized forest, because its size grows with the number of indexed flows rather than with the number of classes. The Random Forest remains the faster classifier per flow, so the trade-off is per-flow latency against adaptability and memory. These results show that retrieval-based classification is a practical and adaptable alternative for real-time encrypted traffic analysis in dynamic network environments.