Computational end-to-end and super-resolution methods to improve thermal infrared remote sensing for agriculture

• Published in: Precision agriculture 2021-04, Vol.22 (2), p.452-474
• Main Authors: Klapp, Iftach, Yafin, Peretz, Oz, Navot, Brand, Omri, Bahat, Idan, Goldshtein, Eitan, Cohen, Yafit, Alchanatis, Victor, Sochen, Nir
• Format: Article
• Language: English
• Subjects: Aerial surveys; Agriculture; Algorithms; Altitude; Artificial neural networks; Atmospheric Sciences; Biomedical and Life Sciences; Cameras; Chemistry and Earth Sciences; Computer applications; Computer Science; Computer simulation; High resolution; High temperature; Infrared cameras; Interpolation; Irrigation water; Life Sciences; Low altitude; Neural networks; Physics; Remote sensing; Remote Sensing/Photogrammetry; Root-mean-square errors; Sharpness; Soil Science & Conservation; Spatial discrimination; Spatial resolution; Statistics for Engineering; Water deficit; Water stress
• ISSN: 1385-2256; 1573-1618
• DOI: 10.1007/s11119-020-09746-y

Increasing global water deficit and demand for yield improvement call for high-resolution monitoring of irrigation, crop water stress, and crops' general condition. To provide high spatial resolution with high-temperature accuracy, remote sensing is conducted at low altitudes using radiometric longwave thermal infrared cameras. However, the radiometric cameras' price, and the low altitude leading to low coverage in a given time, limit the use of radiometric aerial surveys for agricultural needs. This paper presents progress toward solving both limitations using algorithmic and computational imaging methods: stabilizing the readout of low-cost thermal cameras to obtain radiometric data, and improving the latter's low resolution by applying convolutional neural network-based super-resolution. The two methods were merged by an end-to-end algorithm pipeline, providing a large mosaicked image of the field. First, the potential capabilities of a joint estimation method to correct unknown offset and gain were simulated on remotely sensed agricultural data. Comparison to ground-truth measurements showed radiometric accuracy with a root mean square error (RMSE) of 1.3 °C to 1.8 °C. Then, the proposed super-resolution method was demonstrated on experimental and simulated remotely sensed agricultural data. Preliminary experimental results showed 50% improvement in image sharpness relative to bicubic interpolation. The performance of the algorithm was evaluated on 22 simulated cases at × 2 and × 4 magnification. Finally, image mosaicking using the proposed pipeline was demonstrated. A mosaicked image composed of sub-images pre-processed by the proposed computational methods resulted in a RMSE in temperature of 0.8 °C, as compared to 8.2 °C without the initial processing.