Specular highlight removal of light field image combining dichromatic reflection with exemplar patch filling

• Published in: Optics and lasers in engineering 2024-07, Vol.178, p.108175, Article 108175
• Main Authors: Feng, Wei, Sun, Jichen, Liu, Qianqian, Li, Xingang, Liu, Da, Zhai, Zhongsheng
• Format: Article
• Language: English
• Subjects: Color difference; Dichromatic reflection model; Highlight removal; Image gradient; Light field image
• ISSN: 0143-8166; 1873-0302
• DOI: 10.1016/j.optlaseng.2024.108175

•Our method detects and removes specular highlight effectively.•Gaussian mixture model clustering is more accurate than k-means clustering.•Confidence strategy makes the calculated unsaturated pixels more realistic.•New exemplar patch matching criterion finds the best exemplar patch quickly. In this paper, a new method based on dichromatic reflection model (DRM) and exemplar patch is proposed to remove highlight pixels for light field images. Firstly, a Gaussian mixture model clustering method with strong generalization performance combined with depth information is used to classify saturated highlight pixels and unsaturated highlight pixels. A confidence strategy based on DRM is proposed to remove unsaturated highlight pixels, and an exemplar patch matching method based on gradient combined with the sum of square of color difference is designed to remove saturated highlight pixels. Meanwhile, a method named SSIME based on information entropy with structural similarity index measure is designed to quantitatively evaluate the effectiveness of proposed method. Experiments show that our proposed method not only effectively detects and removes specular highlights, but also applies to objects with a large range of highlights, which breakthroughs the limitation that large-region specular highlight cannot be removed due to the short baseline of the light field camera.