A Time Series Self-Supervised Contrastive Pretraining Method With Data Augmentation Using Discrepancy of Reconstruction Information Loss

Self-supervised pretraining has shown considerable advantages in multiple time series tasks. A widely used class of methods is based on contrastive learning and a key problem is how to construct augmented samples. At present, explicit augmentation approaches are the mainstream for constructing positive pairs with similar features. However, explicitly augmented samples may deviate from the original data space and have aliasing features. Therefore, this article designs an end-to-end time series self-supervised contrastive pretraining network with implicit data augmentation using discrepancy of reconstruction information loss (CLR-DRL). Specifically, augmented positive pairs with discrepancies and inherent features of the original series can be generated automatically using different autoencoders (AE) due to different information loss in their hidden spaces. Combined with the joint constraint, inherent features can be extracted and the aliasing impact existing in explicit augmented approaches could be mitigated. Moreover, theoretical analysis is provided using gradient flow calculation, which can also be confirmed by experiment analysis. Experimentally, the CLR-DRL achieves state-of-the-art (SOTA) performance compared to current baselines on in- and cross-domain settings on nine real-world benchmarks for both time series anomaly detection (AD) and classification tasks. An actual pipeline transportation status monitoring case is provided to verify the effectiveness.