Deep learning-based restoration of nonlinear motion blurred images for plant classification using multi-spectral images

• Published in: Applied soft computing 2024-09, Vol.162, p.111866, Article 111866
• Main Authors: Batchuluun, Ganbayar, Hong, Jin Seong, Kim, Seung Gu, Kim, Jung Soo, Park, Kang Ryoung
• Format: Article
• Language: English
• Subjects: Deep learning; Image classification; Multi-spectral images; Nonlinear motion deblurring; Visible light and thermal images of plant
• ISSN: 1568-4946; 1872-9681
• DOI: 10.1016/j.asoc.2024.111866

There have been various plant image-based studies for segmentation, deblurring, super-resolution reconstruction, and classification. However, nonlinear motion blur in thermal images was not considered in the existing studies on plant classification. Nonlinear motion blur occurs in images due to camera or plant movements, and it causes the degradation of plant classification accuracy. Moreover, nonlinear motion blur in images gets worse when both camera and plant movements occur simultaneously. In this case, it becomes difficult to recognize plants, and the performance of plant image classification becomes very low. Therefore, to reduce the nonlinear motion blur, a thermal and visible light plant images-based deblurring network (TVPD-Net) is proposed in this study. In addition, a thermal and visible light plant images-based classification network (TVPC-Net) is also proposed to improve the plant classification performance on deblurred images. Experimental results revealed that the proposed TVPD-Net achieved 21.21 and 22.53 of the peak signal-to-noise ratio (PSNR), and 0.726 and 0.737 of the structural similarity index measure (SSIM) on both visible light and thermal plant image datasets which were self-collected, respectively. Moreover, the proposed TVPC-Net with deblurred images by TVPD-Net achieved 92.52 % (top-1 accuracy) and 87.73 % (harmonic mean of precision and recall (F1-score)). In addition, the experimental results on an open dataset named Hyperspectral Flower Dataset (HFD100) revealed that the proposed plant classification method achieved 90.94 % of top-1 accuracy and 86.21 % of F1-score. The accuracies of the proposed methods are greater than those of the state-of-the-art methods. •This is the first study for plant thermal image-based deblurring.•This is the first study of plant thermal and visible images-based classification with motion blur.•We propose TVPD-Net including 10 residual blocks for plant image deblurring.•We propose TVPC-Net including 4 groups of layers for plant image classification.•Proposed TVPD-Net and TVPC-Net with self-collected database are made open via Github site.