Contrastive learning of defect prototypes under natural language supervision

• Published in: Advanced engineering informatics 2024-10, Vol.62, p.102749, Article 102749
• Main Authors: Cheng, Huyue, Jiang, Hongquan, Yan, Haobo, Zhang, Wanjun
• Format: Article
• Language: English
• Subjects: Defect recognition; Expert knowledge; Interpretable machine learning; Multimodal deep learning; Prototype learning
• ISSN: 1474-0346
• DOI: 10.1016/j.aei.2024.102749

The use of machine learning theory for decision-making in manufacturing systems has become a convenient way to improve manufacturing efficiency. One task in this domain is to recognize girth weld defects based on ray images, and incorporating radiographic inspection industry specifications and the experiences of specialists in the model design process has become challenging. The emergence of image-text multimodal models confirms that natural language can supervise a much wider set of visual concepts; accordingly, the experiences of professionals in defect recognition work are summarized and generalized in this paper. Domain experience prototypes of different types of defects are established in text form, and the contrastive learning method of image and text feature matching is used to achieve girth weld defect recognition. Finally, the proposed method is validated on the radiographic inspection data of in-service pipelines, which demonstrates that the proposed method can be used to recognize defects according to the domain specification and produces better outcomes than common type recognition networks.