Adaptive and automatic aerial image restoration pipeline leveraging pre-trained image restorer with lightweight Fully Convolutional Network

• Published in: Expert systems with applications 2025-01, Vol.259, p.125210, Article 125210
• Main Authors: Hossain, Md Yearat, Rakib, Md Mahbub Hasan, Rajit, Shafayet, Nijhum, Ifran Rahman, Rahman, Rashedur M.
• Format: Article
• Language: English
• Subjects: Automatic enhancement; Deep learning; Image restoration; Remote sensing; Satellite imagery
• ISSN: 0957-4174
• DOI: 10.1016/j.eswa.2024.125210

Earth observation and remote sensing necessitate the use of aerial imaging, and unmanned aerial vehicles (UAVs) are widely utilized for this purpose. Although UAVs can be costly, using inexpensive components could result in issues such as noise, defocusing, and motion blur. Utilizing advanced deep learning algorithms, these images can be effectively restored. However, this necessitates the use of physical effort since each image must be restored manually. Our work introduces a new pipeline that can automatically categorize the degradation types of aerial images and subsequently apply the necessary restoration techniques on degraded images. A lightweight adaptive degradation classifier model was proposed to handle input from images of varying sizes and provide guidance to a image restoration model for achieving the desired task. The degradation classifier is a Fully Convolutional Network capable of categorizing images of any size into 3 distinct classes. It offers guidance to the subsequent image restorer. The restoration model is a pre-trained transformer-based model that efficiently restores different types of image degradations. In addition, we have implemented a pipeline loss to assess the performance of our complete system. To build the system, a dataset is created by extracting cropped patches from high-resolution images obtained through Google Earth. Subsequently, the clean patches were artificially deteriorated by applying different filters to replicate the image degradations. The degradation classifier achieved a mean accuracy of 96.71% across 6 different test scenarios, with a standard deviation of 2.97. The mean pipeline loss was 24.95, with a standard deviation of 1.79. Furthermore, we showcased the efficacy of our pipeline in a practical implementation involving aerial imagery, specifically in the domain of vehicle detection. We conducted training on an object detection model and then evaluated the detection performance on both degraded and automatically restored images. Our experiments demonstrate the efficient performance of our method in automatically restoring degraded aerial images, even though the restoration models were not specifically trained on aerial images. In addition, we examined current limitations and difficulties in creating such systems and offered comprehensive perspectives on future advancements. •Inexpensive UAVs may suffer from various image quality degradations.•Large pre-trained image restorers can be utilized but requir