Learning Converged Propagations With Deep Prior Ensemble for Image Enhancement

• Published in: IEEE transactions on image processing 2019-03, Vol.28 (3), p.1528-1543
• Main Authors: Liu, Risheng, Ma, Long, Wang, Yiyang, Zhang, Lei
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Control theory; Convergence; Cues; Feedback control; Image enhancement; Image restoration; Investigations; Neural networks; non-convex optimization; Optimization; prior model; Propagation; residual CNN; Task analysis; Visual perception; visual propagation; Visualization
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2018.2875568

Enhancing the visual qualities of images plays very important roles in various vision and learning applications. In the past few years, both knowledge-driven maximum a posterior (MAP) with prior modelings and fully data-dependent convolutional neural network (CNN) techniques have been investigated to address specific enhancement tasks. In this paper, by exploiting the advantages of these two types of mechanisms within a complementary propagation perspective, we propose a unified framework, named deep prior ensemble (DPE) for solving various image enhancement tasks. Specifically, we first establish the basic propagation scheme based on the fundamental image modeling cues and then introduce residual CNNs to help predicting the propagation direction at each stage. By designing prior projections to perform feedback control, we theoretically prove that even with experience-inspired CNNs, DPE is definitely converged and the output will always satisfy our fundamental task constraints. The main advantage against the conventional optimization-based MAP approaches is that our descent directions are learned from collected training data, thus are much more robust to unwanted local minimums. While, compared with existing CNN type networks, which are often designed in heuristic manners without theoretical guarantees, DPE is able to gain advantages from rich task cues investigated on the bases of domain knowledges. Therefore, the DPE actually provides a generic ensemble methodology to integrate both knowledge and data-based cues for different image enhancement tasks. More importantly, our theoretical investigations verify that the feed-forward propagations of DPE are properly controlled toward our desired solution. Experimental results demonstrate that the proposed DPE outperforms the state-of-the-arts on a variety of image enhancement tasks in terms of both quantitative measure and visual perception quality.