Image Visual Realism: From Human Perception to Machine Computation

• Published in: IEEE transactions on pattern analysis and machine intelligence 2018-09, Vol.40 (9), p.2180-2193
• Main Authors: Fan, Shaojing, Ng, Tian-Tsong, Koenig, Bryan Lee, Herberg, Jonathan Samuel, Jiang, Ming, Shen, Zhiqi, Zhao, Qi
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Benchmark testing; Computation; Computational modeling; Computer Graphics; convolutional neural network; Fabric analysis; Face; Female; human psychophysics; Humans; Judgments; Machine Learning; Male; Mathematical models; Models, Statistical; Neural Networks (Computer); Psychophysics - methods; Realism; Rendering (computer graphics); Retouching; Solid modeling; statistical modeling; Statistical models; Video boards; Virtual Reality; Visual Perception - physiology; Visual perception driven algorithms; Visual realism; Visualization
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2017.2747150

Visual realism is defined as the extent to which an image appears to people as a photo rather than computer generated. Assessing visual realism is important in applications like computer graphics rendering and photo retouching. However, current realism evaluation approaches use either labor-intensive human judgments or automated algorithms largely dependent on comparing renderings to reference images. We develop a reference-free computational framework for visual realism prediction to overcome these constraints. First, we construct a benchmark dataset of 2,520 images with comprehensive human annotated attributes. From statistical modeling on this data, we identify image attributes most relevant for visual realism. We propose both empirically-based (guided by our statistical modeling of human data) and deep convolutional neural network models to predict visual realism of images. Our framework has the following advantages: (1) it creates an interpretable and concise empirical model that characterizes human perception of visual realism; (2) it links computational features to latent factors of human image perception.