Fusion of multi-source retinal fundus images via automatic registration for clinical diagnosis

• Published in: Neurocomputing (Amsterdam) 2021-10, Vol.459, p.370-382
• Main Authors: Dan, Tingting, Hu, Yu, Han, Chu, Fan, Zhihao, Huang, Zhuobin, Zhang, Bin, Tao, Guihua, Liu, Baoyi, Yu, Honghua, Cai, Hongmin
• Format: Article
• Language: English
• Subjects: Adjustable threshold selection; Geometric structure constraint; Multiple features; Multiple sources; Retinal fundus image registration
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2021.05.091

•The registration of retinal images suffering adverse conditions remains challenging.•Dual correspondences and geometric constraints are employed jointly to guild registration.•The proposed automatic method is promising in identifying and monitoring of retinal diseases. Diabetic retinopathy, age-related macular degeneration and glaucoma, are the leading causes of visual impairment or blindness of the population across different ages. Retinal fundus imaging is a clinically regular tool for the diagnosis of retinal diseases. In the interest of having a comprehensive understanding of the fundus condition, it is valuable to leverage multiple fundus images from different modalities. However, a direct fusion of the multi-source fundus images eases to mis-align the physiological structure or spatial position due to possible eyeball rotations or head movements. The problem turns out to be more severe if the images were corrupted by ill conditions on eyes, such as micro-bleeding and plaques. To tackle this problem, we propose a multi-source registration model for retinal fundus images. Our proposed method considers multiple correspondences and dual structural constraints during the registration process. The method firstly selects adequate feature points by an adjustable threshold selection strategy. Then a feature-guided correspondence estimation model is established to build complementary features. Finally, their spatial transformation is built by using mean shift evolution. The evolution is guided by Tikhonov regularization on dual geometric structures. It overcomes the mess of mean shift vector field and mitigating the ill-posed displacement in field recovery. We have conducted our method on the collected 220 multi-source retinal fundus image pairs, which involve minor and larger displacement or severe retinopathy lesions, as well as additive different intensities of Gaussian noises. Extensive experiments demonstrate that the proposed method consistently outperforms seven feature-based methods.