Robust Image Completion via Deep Feature Transformations

• Published in: IEEE access 2019, Vol.7, p.113916-113930
• Main Authors: Jiang, Jianmin, Kasem, Hossam M., Hung, Kwok-Wai
• Format: Article
• Language: English
• Subjects: Computer vision; Deep feature transformation; Deep learning; Distortion; Feature extraction; geometric transformation; image completion; Image quality; Image reconstruction; Interpolation; Robustness; Spatial resolution; Transformations
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2019.2935130

For many practical applications, it is essential to address both geometric corrections and missing information reconstruction of face images and natural images. However, it is unfavorable to separate the problem into two sub-tasks due to error accumulations of sequential tasks. In this paper, we propose a novel robust missing information reconstruction framework via deep feature transformations to simultaneously address both geometric corrections and image completion. Specifically, our proposed framework realizes multiple channel spatial transformations to tackle geometric corrections, and address image completion through non-linear features projections. The flow of our framework includes deep feature extraction, feature enhancement, feature projection, and feature refinement, where deep features are extracted and learnt to achieve robust image completion. Experimental results show the superior performance of our framework for both face images and natural images in various databases. Compared with the conventional approaches approach to split the problem into two sub-tasks, including image inpainting and spatial transformation, our proposed framework achieves a number of advantages, including i) an unified framework to automatically correct the geometric distortions and to reconstruct the missing information simultaneously and ii) achieving much better visual quality for those recovered images.