Improvement of the reading algorithm for pointer instruments based on deep learning

Abstract Pointer instruments, widely used in complex environments, present significant challenges for manual reading due to inefficiency and difficulty. For intelligent devices, problems arise with instrument localization, low inference accuracy, and incorrect readings. To address these issues, this paper proposes a novel deep-learning algorithm that divides the reading task of pointer instruments into three stages. First, the NV-YOLOv8 model is introduced to identify the instrument’s location, and Spatial Transformer Networks are employed to correct spatial transformations of the instrument panel. Then, a U-Net-based improved algorithm is used to segment the instrument panel and pointer. Finally, an end-to-end text recognition method, along with a novel weighted angle approach, is applied to accurately read the instrument values. To evaluate the reading accuracy, we introduce the mean relative error metric, and our algorithm is assessed on a newly collected, The diverse dataset consists of 1,500 images of various instruments taken from different angles and lighting conditions under challenging conditions such as tilting, blurring and panel damage. These include pressure gauges, temperature gauges, hydraulic gauges, and more. This diverse dataset plays a crucial role in improving the robustness of the algorithm, enabling it to excel in a wide range of real-world scenarios.The experimental results demonstrate that our algorithm achieves a mean relative error of only 2.76%, outperforming other mainstream methods by a significant margin. This confirms the promising performance of the proposed method in enhancing the accuracy and robustness of pointer instrument reading, even under challenging conditions.