Deep learning-based autonomous real-time digital meter reading recognition method for natural scenes

• Published in: Measurement : journal of the International Measurement Confederation 2023-11, Vol.222, p.113615, Article 113615
• Main Authors: Peng, Jianqing, Zhou, Wei, Han, Yu, Li, Mengtang, Liu, Wanquan
• Format: Article
• Language: English
• Subjects: Autonomous recognition; Deep learning; Digital meter reading; Keypoint detection; Perspective transformation
• ISSN: 0263-2241; 1873-412X
• DOI: 10.1016/j.measurement.2023.113615

[Display omitted] •A real-time digital meter reading recognition system in natural scenes is proposed;•A YOLO-style corner point detection method for reading area is proposed;•A YOLO-style end-to-end reading recognition method is proposed;•The digital meter reading recognition method has high accuracy and practicality;•A challenging dataset for digital meter readings was produced. Natural scenes with variable illumination, variable target scale and angular tilt pose significant challenges to the autonomous recognition of digital meter readings. Based on this, this paper proposes a deep learning-based autonomous real-time digital meter reading recognition method for natural scenes. First, the YOLO-style corner point detection method (YOLO-CPDM) for the reading area is proposed by reconstructing the detection heads and incorporating the corner detection loss function. Its localization accuracy is further refined by embedding attention mechanism module, implementing dynamic loss function and enhancing training data diversity through offline augmentation techniques like image rotation and flipping. Then, the detected corner points are used to geometrically correct the distorted reading area by perspective transformation to mitigate the interference caused by the shooting angle. Next, the YOLO-style end-to-end reading recognition method (YOLO-EERRM) is proposed to accurately extract the characters in the reading area. Finally, the validity of the YOLO-CPDM and YOLO-EERRM was verified on a produced dataset named SYSU-DM and 2 public datasets. Compared with the State of the arts (SOTA) keypoint detection model, the mean Average Precision @ 50:95 scores of the YOLO-CPDM improved by 2.8, 4.1, and 1.1 points, respectively, while the inference latency was only 5.3 ms, and YOLO-EERRM achieved 100 % accuracy and 3.1 ms inference latency on the SYSU-DM dataset. Statistically, the complete digital meter reading recognition method has 99.6 % accuracy and 8.6 ms inference latency, indicating that the system has high recognition accuracy and practicality.