Pix2Pix Network to Estimate Agricultural Near Infrared Images from RGB Data

• Published in: Canadian journal of remote sensing 2022-03, Vol.48 (2), p.299-315
• Main Authors: de Lima, Daniel Caio, Saqui, Diego, Mpinda, Steve Ataky Tsham, Saito, José Hiroki
• Format: Article
• Language: English
• ISSN: 0703-8992; 1712-7971
• DOI: 10.1080/07038992.2021.2016056

Remote sensing has been applied to agriculture, making it possible to acquire a large amount of data far away from crops, providing information for decision making by producers that can impact production costs and crops quality. One way of getting the production information is through vegetation indices, arithmetic operations that use spectral bands, especially the Near Infrared (NIR). However, sensors that capture this spectral information are very expensive for small producers to afford it. In a previous article, a pixel-to-pixel image synthesis model to estimate NIR images from RGB data using hyperspectral endmembers (pure hyperspectral signatures) was described. In this work, an image-to-image synthesis model, known as Pix2Pix, is used for estimating NIR images from low-cost RGB camera images. Pix2Pix is a kind of Generative Adversarial Networks (GANs), composed by two neural networks, a generator (G) and a discriminator (D), that compete. G learns to create images from a random noise inputs and D learns to verify if these images are real or fake. The results showed that the presented method generated NIR images quite similar to real ones, reaching a value of 0.912 on M3SIM similarity metric, outperforming results obtained with the previous endmembers method (0.775 on M3SIM).