Unsupervised end-to-end multiscale neural network for multi-focus MicroLED image fusion

• Published in: Physica scripta 2024-10, Vol.99 (10), p.106001
• Main Authors: Yu, Wenlin, Chen, Jinbiao, Li, Cheng
• Format: Article
• Language: English
• Subjects: MicroLED; multi-focus image fusion; two-stage training strategy; unsupervised
• ISSN: 0031-8949; 1402-4896
• DOI: 10.1088/1402-4896/ad7064

MicroLED has a broad application prospect in visible light communication, medical detection, and other fields, owing to its small size, high integration, and long service life. However, capturing a full-focus image during microscopic visual inspection of MicroLED is challenging due to the significant thickness of the chip. To address this problem, an end-to-end neural network named MMLFuse is proposed for MicroLED image fusion, which uses unsupervised learning to directly generate fused images from two original images. Firstly, we introduce the Spatial Pyramid Pooling Mixing (SPPM) module for rapid extraction of partially focused image features. The extracted features are then used to obtain a weight map, which is further refined using a moving window smoothing technique. This refined weight map is employed for feature fusion, and the fused image is reconstructed based on the fused features. Specifically, the network uses a two-stage training strategy with different loss functions for each stage to improve the convergence speed of the model and the quality of the fused image. In particular, mask loss is designed in the second stage to ensure that the network pays more attention to the focus area during training to accurately match the corresponding input image. Experimental results demonstrate that MMLFuse achieves superior performance on the MicroLED dataset compared to other classical methods, highlighting its effectiveness and potential in the field.