Steel product number recognition framework using semantic mask-conditioned diffusion model with limited data

• Published in: Journal of industrial information integration 2024-03, Vol.38, p.100559, Article 100559
• Main Authors: Choi, Hyeyeon, Yun, Jong Pil, Kim, Bum Jun, Jang, Hyeonah, Shin, WooSang, Kim, Sang Woo
• Format: Article
• Language: English
• Subjects: Computer vision; Diffusion models; Image generation; Limited data; Object recognition; Product management
• ISSN: 2452-414X
• DOI: 10.1016/j.jii.2024.100559

Steel product number recognition (SPNR) is crucial for efficient product management in the steel industry, and there has been a recent focus on utilizing number recognition algorithms to pursue the automation of product management. Deep learning-based methods have enhanced the performance of SPNR. However, the issue of data scarcity has been overlooked in previous studies. In actual industrial environments, data related to product numbers are scarce, owing to frequently changing environmental factors such as illumination, dust, heating surfaces, and the seasonality of such numbers. This lack of data significantly influences the accuracy of SPNR. To address the issue of data scarcity, this paper proposes an SPNR framework that utilizes images generated from the proposed semantic mask-conditioned diffusion model (SMDM). First, we designed an SMDM architecture comprising encoding parts for the font style and text format based on a diffusion model. Second, the SMDM was trained to generate product number images with desired font styles, text formats, and contents. Finally, the generated images were utilized as training data for the SPNR model. Extensive experiments on three distinct types of real-world datasets demonstrate that the proposed framework yielded significantly higher SPNR accuracy compared with those of existing methods. The experimental results also showed that the SMDM could generate out-of-distribution samples that were not included in the distribution of the small training dataset, thereby improving the distribution diversity of the training data. By addressing the data scarcity problem, our framework can aid in advancing the application of deep learning-based algorithms in the steel industry. •The problem of recognizing product numbers with limited data is addressed.•Product number image containing user defined characteristics can be generated.•Proposed SMDM generates out-of-distribution samples.•Generated samples diversify the data distribution and improve recognition accuracy.