Estimation of Degradation Degree in Road Infrastructure Based on Multi-Modal ABN Using Contrastive Learning

• Published in: Sensors (Basel, Switzerland) Switzerland), 2023-02, Vol.23 (3), p.1657
• Main Authors: Higashi, Takaaki, Ogawa, Naoki, Maeda, Keisuke, Ogawa, Takahiro, Haseyama, Miki
• Format: Article
• Language: English
• Subjects: Annotations; Artificial intelligence; Classification; contrastive learning; convolutional neural network; Datasets; Deep learning; distress image classification; Image classification; Image interpretation, Computer assisted; Infrastructure; Maintenance and repair; Medical research; Methods; multi-modal learning; Neural networks; Remote sensing; Representations; Roads; Roads & highways; self-supervised learning; Unsupervised learning
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s23031657

This study presents a method for distress image classification in road infrastructures introducing self-supervised learning. Self-supervised learning is an unsupervised learning method that does not require class labels. This learning method can reduce annotation efforts and allow the application of machine learning to a large number of unlabeled images. We propose a novel distress image classification method using contrastive learning, which is a type of self-supervised learning. Contrastive learning provides image domain-specific representation, constraining such that similar images are embedded nearby in the latent space. We augment the single input distress image into multiple images by image transformations and construct the latent space, in which the augmented images are embedded close to each other. This provides a domain-specific representation of the damage in road infrastructure using a large number of unlabeled distress images. Finally, the representation obtained by contrastive learning is used to improve the distress image classification performance. The obtained contrastive learning model parameters are used for the distress image classification model. We realize the successful distress image representation by utilizing unlabeled distress images, which have been difficult to use in the past. In the experiments, we use the distress images obtained from the real world to verify the effectiveness of the proposed method for various distress types and confirm the performance improvement.