Large-Scale Crowdsourced Study for Tone-Mapped HDR Pictures

• Published in: IEEE transactions on image processing 2017-10, Vol.26 (10), p.4725-4740
• Main Authors: Kundu, Debarati, Ghadiyaram, Deepti, Bovik, Alan C., Evans, Brian L.
• Format: Article
• Language: English
• Subjects: Algorithm design and analysis; Algorithms; Color; Consumers; Correlation analysis; Crowdsourcing; Dynamic range; Exposure; high dynamic range; Human performance; Image coding; Image quality; Image quality assessment; Mapping; Observers; Operators (mathematics); Pictures; Pixels; Post-processing; Quality; Quality assessment; subjective study; Viewing
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2017.2713945

Measuring digital picture quality, as perceived by human observers, is increasingly important in many applications in which humans are the ultimate consumers of visual information. Standard dynamic range (SDR) images provide 8 b/color/pixel. High dynamic range (HDR) images, usually created from multiple exposures of the same scene, can provide 16 or 32 b/color/pixel, but need to be tonemapped to SDR for display on standard monitors. Multiexposure fusion (MEF) techniques bypass HDR creation by fusing an exposure stack directly to SDR images to achieve aesthetically pleasing luminance and color distributions. Many HDR and MEF databases have a relatively small number of images and human opinion scores, obtained under stringently controlled conditions, thereby limiting realistic viewing. Moreover, many of these databases are intended to compare tone-mapping algorithms, rather than being specialized for developing and comparing image quality assessment models. To overcome these challenges, we conducted a massively crowdsourced online subjective study. The primary contributions described in this paper are: 1) the new ESPL-LIVE HDR Image Database that we created containing diverse images obtained by tonemapping operators and MEF algorithms, with and without postprocessing; 2) a large-scale subjective study that we conducted using a crowdsourced platform to gather more than 300000 opinion scores on 1811 images from over 5000 unique observers; and 3) a detailed study of the correlation performance of the state-ofthe-art no-reference image quality assessment algorithms against human opinion scores of these images. The database is available at http://signal.ece.utexas.edu/%7Edebarati/HDRDatabase.zip.