Self-Adaptive Sampling Based Per-Flow Traffic Measurement

Per-flow traffic measurement in the high-speed network plays an important role in many practical applications. Due to the limited on-chip memory and the mismatch between off-chip memory speed and line rate, sampling-based methods select and forward a part of flow traffic to off-chip memory, which complements sketch-based solutions in estimation accuracy and online query support. However, most current work uses the same sampling probability for all flows, leading to the waste in storage and communication resources. In practice, different flows often require different sampling rates to meet the same accuracy constraint. This paper presents self-adaptive sampling, a framework to sample each flow with a probability adapted to flow size/spread. Then we propose three algorithms, SAS-LC, SAS-LOG, and SAS-HYB. SAS-LC and SAS-LOG are geared towards per-flow spread estimation and per-flow size estimation by using different compression functions. SAS-HYB combines the advantages of SAS-LC and SAS-LOG, showing higher efficiency when both small flows and large flows are interested. We implement our estimators in hardware using NetFPGA. Experimental results based on real Internet traces show that, compared to the state-of-the-art in per-flow spread estimation, SAS-LC can save around 10% on-chip space and reduce up to 40% communication cost for large flows. In per-flow size estimation, SAS-LOG can save 40% on-chip space and reduce up to 96% communication costs for large flows. Moreover, SAS-HYB's on-chip memory usage will not be larger than SAS-LC or SAS-LOG and can save up to 19% on-chip space than SAS-LOG when both small flows and large flows are interested.