In situ grape ripeness estimation via hyperspectral imaging and deep autoencoders

• Published in: Computers and electronics in agriculture 2023-09, Vol.212, p.108098, Article 108098
• Main Authors: Tsakiridis, Nikolaos L., Samarinas, Nikiforos, Kokkas, Stylianos, Kalopesa, Eleni, Tziolas, Nikolaos V., Zalidis, George C.
• Format: Article
• Language: English
• Subjects: Deep learning; Dissolved solids; Grape cultivars; Vis–NIR; VNIR
• ISSN: 0168-1699; 1872-7107
• DOI: 10.1016/j.compag.2023.108098

The estimation of the grapes’ maturity in the field using non-destructive techniques is of high interest for the high-valued vinified grapes, particularly towards the development of fully automated agrobots that perform selective harvesting operations. Whereas infrared spectroscopy has been employed using point spectrometers in the laboratory and in the field, imaging spectrometers have mainly been tested in controlled laboratory conditions due to issues with varying illumination. In this paper, the application of the autoencoder framework is proposed, which is employed to transform the raw recorded spectra, regardless of illumination conditions, into standardized reflectance spectra; thus addressing the inherent difficulties which hamper the direct application of hyperspectral imaging in the field. To validate the methodology, the sugar content (∘Brix) of four grape varieties, namely Chardonnay, Malagouzia, Sauvignon-Blanc, and Syrah, is estimated. Two different autoencoder architectures are examined: deep fully-connected (DAE) and deep convolutional autoencoders (DCAE), while the estimation of sugar content takes place using as input both from the encoded (latent) space and from the autoencoders’ output, i.e., the transformed standardized spectra. The use of multiple spectral pre-treatments is further examined to enhance the accuracy of prediction. Despite that DAE and DCAE showcase comparable similarity metrics, DCAE statistically outperforms DAE when using both the encoded space and the autoencoders’ output, attesting to the suitability of the convolutional autoencoder framework. On the other hand, there is no statistical significant difference when employing multiple input pre-treatments. The accuracy of estimation (mean RMSE 1.83 ∘Brix, R2 0.70, RPIQ 2.43) is comparable to other studies that directly work with standardized reflectance spectra in laboratory conditions. •In situ hyperspectral imaging faces the varying illumination problem of the targets.•The autoencoder framework is proposed to address this issue and different architectures are investigated.•The presented solution can sufficiently transform raw spectra into standardized reference spectra.•The proposed framework is applied for grape ripeness estimation with fair results.