Threshold-free anomaly detection for rotational machinery: A probabilistic framework based on VAE and KDE to mitigate false alarms

Anomaly detection in rotating machinery often faces challenges in threshold determination and false alarms, which not only restrict the generalization ability of models but also lead to unnecessary downtime. To address this issue, this study explores a probabilistic expression framework suitable for anomaly detection of rotating machinery under constant-speed and stable operating conditions. Specifically, the reconstruction error (RE) based on Variational Autoencoder (VAE) is first adopted as a health indicator. Aiming at the problems of threshold setting and false alarms in its application, a probabilistic expression framework based on RE and Kernel Density Estimation (KDE) is proposed. This framework can directly provide anomaly probability density, and distinguish between interference and anomalies by quantifying the impact of abnormal values on the shape of the probability density curve, thereby avoiding threshold dependency and the risk of false alarms. The performance of this indicator is validated using experimental data of gears and bearings. The empirical relationship between the coefficient of variation (CV) of reconstruction error and the KDE window h is analyzed and presented, and a comparative analysis with traditional methods is also conducted. The research shows that this framework does not require setting an anomaly detection threshold for reconstruction error, nor does it issue alarms for non-abnormal states, providing a new solution for anomaly detection in rotating machinery.