A Study on Enhancing the Visual Fidelity of Aviation Simulators Using WGAN-GP for Remote Sensing Image Color Correction

• Published in: Applied sciences 2024-10, Vol.14 (20), p.9227
• Main Authors: Lee, Chanho, Kwon, Hyukjin, Choi, Hanseon, Choi, Jonggeun, Lee, Ilkyun, Kim, Byungkyoo, Jang, Jisoo, Shin, Dongkyoo
• Format: Article
• Language: English
• Subjects: Accuracy; Aeronautics; Aviation; aviation simulators; color enhancing; Datasets; Histograms; Methods; Pilots; Remote sensing; remote sensing images; Wasserstein GAN with Gradient Penalty
• ISSN: 2076-3417; 2076-3417
• DOI: 10.3390/app14209227

When implementing outside-the-window (OTW) visuals in aviation tactical simulators, maintaining terrain image color consistency is critical for enhancing pilot immersion and focus. However, due to various environmental factors, inconsistent image colors in terrain can cause visual confusion and diminish realism. To address these issues, a color correction technique based on a Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) is proposed. The proposed WGAN-GP model utilizes multi-scale feature extraction and Wasserstein distance to effectively measure and adjust the color distribution difference between the input image and the reference image. This approach can preserve the texture and structural characteristics of the image while maintaining color consistency. In particular, by converting Bands 2, 3, and 4 of the BigEarthNet-S2 dataset into RGB images as the reference image and preprocessing the reference image to serve as the input image, it is demonstrated that the proposed WGAN-GP model can handle large-scale remote sensing images containing various lighting conditions and color differences. The experimental results showed that the proposed WGAN-GP model outperformed traditional methods, such as histogram matching and color transfer, and was effective in reflecting the style of the reference image to the target image while maintaining the structural elements of the target image during the training process. Quantitative analysis demonstrated that the mid-stage model achieved a PSNR of 28.93 dB and an SSIM of 0.7116, which significantly outperforms traditional methods. Furthermore, the LPIPS score was reduced to 0.3978, indicating improved perceptual similarity. This approach can contribute to improving the visual elements of the simulator to enhance pilot immersion and has the potential to significantly reduce time and costs compared to the manual methods currently used by the Republic of Korea Air Force.