Benchmarking Low-Light Image Enhancement and Beyond

• Published in: International journal of computer vision 2021-04, Vol.129 (4), p.1153-1184
• Main Authors: Liu, Jiaying, Xu, Dejia, Yang, Wenhan, Fan, Minhao, Huang, Haofeng
• Format: Article
• Language: English
• Subjects: Algorithms; Annotations; Artificial Intelligence; Benchmarks; Computer Imaging; Computer Science; Datasets; Detectors; Face recognition; Feature extraction; Image enhancement; Image Processing and Computer Vision; Light; Machine vision; Modules; Optimization; Pattern Recognition; Pattern Recognition and Graphics; Special Issue on Computer Vision for All Seasons: Adverse Weather and Lighting Conditions; Vision
• ISSN: 0920-5691; 1573-1405
• DOI: 10.1007/s11263-020-01418-8

In this paper, we present a systematic review and evaluation of existing single-image low-light enhancement algorithms. Besides the commonly used low-level vision oriented evaluations, we additionally consider measuring machine vision performance in the low-light condition via face detection task to explore the potential of joint optimization of high-level and low-level vision enhancement. To this end, we first propose a large-scale low-light image dataset serving both low/high-level vision with diversified scenes and contents as well as complex degradation in real scenarios, called Vision Enhancement in the LOw-Light condition (VE-LOL). Beyond paired low/normal-light images without annotations, we additionally include the analysis resource related to human, i.e. face images in the low-light condition with annotated face bounding boxes. Then, efforts are made on benchmarking from the perspective of both human and machine visions. A rich variety of criteria is used for the low-level vision evaluation, including full-reference, no-reference, and semantic similarity metrics. We also measure the effects of the low-light enhancement on face detection in the low-light condition. State-of-the-art face detection methods are used in the evaluation. Furthermore, with the rich material of VE-LOL, we explore the novel problem of joint low-light enhancement and face detection. We develop an enhanced face detector to apply low-light enhancement and face detection jointly. The features extracted by the enhancement module are fed to the successive layer with the same resolution of the detection module. Thus, these features are intertwined together to unitedly learn useful information across two phases, i.e. enhancement and detection. Experiments on VE-LOL provide a comparison of state-of-the-art low-light enhancement algorithms, point out their limitations, and suggest promising future directions. Our dataset has supported the Track “Face Detection in Low Light Conditions” of CVPR UG2+ Challenge (2019–2020) ( http://cvpr2020.ug2challenge.org/ ).