Loading…
A Novel Rank Learning Based No-Reference Image Quality Assessment Method
Recently, applying deep learning to no-reference image quality assessment (NR-IQA) has received significant attention. Especially in the last five years, an increasing interest has been drawn to the studies of rank learning since it can help mitigate the problem of small IQA datasets. However, on on...
Saved in:
Published in: | IEEE transactions on multimedia 2022-01, Vol.24, p.4197-4211 |
---|---|
Main Authors: | , , , , |
Format: | Article |
Language: | English |
Subjects: | |
Citations: | Items that this one cites Items that cite this one |
Online Access: | Get full text |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
cited_by | cdi_FETCH-LOGICAL-c206t-28de292e0f220105966cb3e0c678d202a8fe5ddfcf47ccda116188cfb88610d23 |
---|---|
cites | cdi_FETCH-LOGICAL-c206t-28de292e0f220105966cb3e0c678d202a8fe5ddfcf47ccda116188cfb88610d23 |
container_end_page | 4211 |
container_issue | |
container_start_page | 4197 |
container_title | IEEE transactions on multimedia |
container_volume | 24 |
creator | Ou, Fu-Zhao Wang, Yuan-Gen Li, Jin Zhu, Guopu Kwong, Sam |
description | Recently, applying deep learning to no-reference image quality assessment (NR-IQA) has received significant attention. Especially in the last five years, an increasing interest has been drawn to the studies of rank learning since it can help mitigate the problem of small IQA datasets. However, on one hand, existing rank learning is not suitable for the authentically distorted images due to the lack of generated rank samples. On the other hand, the output of existing rank loss functions is uncontrollable, resulting in reduced performance. Motivated by these two limitations, we propose a novel rank learning based NR-IQA method, termed controllable list-wise ranking IQA (CLRIQA) in this paper. To be specific, we first present an imaging-heuristic approach, in which the over- and under-exposure is formulated as an inverse of the Weber-Fechner law, and fusion strategy and compression are adopted, to simulate the authentic distortion and generate the rank image samples. These samples are label-free yet associated with quality ranking information. Then we design a controllable list-wise ranking (CLR) loss function by setting an upper and lower bound of rank range and introducing an adaptive margin to tune rank interval. Finally, both the generated rank samples and proposed CLR are used to pre-train a convolutional neural network. Moreover, to obtain a more accurate prediction model, we take advantage of the IQA datasets to fine-tune the pre-trained network further. Various experiments are conducted on the IQA benchmark datasets, and experimental results demonstrate the effectiveness of the proposed CLRIQA method. The source code and network model can be downloaded at the following web address: https://github.com / GZHU-DVL / CLRIQA . |
doi_str_mv | 10.1109/TMM.2021.3114551 |
format | article |
fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2712055578</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>9548827</ieee_id><sourcerecordid>2712055578</sourcerecordid><originalsourceid>FETCH-LOGICAL-c206t-28de292e0f220105966cb3e0c678d202a8fe5ddfcf47ccda116188cfb88610d23</originalsourceid><addsrcrecordid>eNo9kNFLwzAQh4MoOKfvgi8Fnzvv0qZJH6eoG2yKYz6HLL3Mzq2dSSfsvzdjw6c7uO93d3yM3SIMEKF8mE-nAw4cBxliLgSesR6WOaYAUp7HXnBIS45wya5CWAFECGSPjYbJW_tL62Rmmu9kQsY3dbNMHk2gKk7SGTny1FhKxhuzpORjZ9Z1t0-GIVAIG2q6ZErdV1tdswtn1oFuTrXPPl-e50-jdPL-On4aTlLLoehSririJSdwnAOCKIvCLjICW0hVxf-NciSqylmXS2srg1igUtYtlCoQKp712f1x79a3PzsKnV61O9_Ek5pL5CCEkCpScKSsb0Pw5PTW1xvj9xpBH3zp6EsffOmTrxi5O0ZqIvrHS5ErxWX2ByCJZNA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2712055578</pqid></control><display><type>article</type><title>A Novel Rank Learning Based No-Reference Image Quality Assessment Method</title><source>IEEE Electronic Library (IEL) Journals</source><creator>Ou, Fu-Zhao ; Wang, Yuan-Gen ; Li, Jin ; Zhu, Guopu ; Kwong, Sam</creator><creatorcontrib>Ou, Fu-Zhao ; Wang, Yuan-Gen ; Li, Jin ; Zhu, Guopu ; Kwong, Sam</creatorcontrib><description><![CDATA[Recently, applying deep learning to no-reference image quality assessment (NR-IQA) has received significant attention. Especially in the last five years, an increasing interest has been drawn to the studies of rank learning since it can help mitigate the problem of small IQA datasets. However, on one hand, existing rank learning is not suitable for the authentically distorted images due to the lack of generated rank samples. On the other hand, the output of existing rank loss functions is uncontrollable, resulting in reduced performance. Motivated by these two limitations, we propose a novel rank learning based NR-IQA method, termed controllable list-wise ranking IQA (CLRIQA) in this paper. To be specific, we first present an imaging-heuristic approach, in which the over- and under-exposure is formulated as an inverse of the Weber-Fechner law, and fusion strategy and compression are adopted, to simulate the authentic distortion and generate the rank image samples. These samples are label-free yet associated with quality ranking information. Then we design a controllable list-wise ranking (CLR) loss function by setting an upper and lower bound of rank range and introducing an adaptive margin to tune rank interval. Finally, both the generated rank samples and proposed CLR are used to pre-train a convolutional neural network. Moreover, to obtain a more accurate prediction model, we take advantage of the IQA datasets to fine-tune the pre-trained network further. Various experiments are conducted on the IQA benchmark datasets, and experimental results demonstrate the effectiveness of the proposed CLRIQA method. The source code and network model can be downloaded at the following web address: https://github.com <inline-formula><tex-math notation="LaTeX">/</tex-math></inline-formula> GZHU-DVL <inline-formula><tex-math notation="LaTeX">/</tex-math></inline-formula> CLRIQA .]]></description><identifier>ISSN: 1520-9210</identifier><identifier>EISSN: 1941-0077</identifier><identifier>DOI: 10.1109/TMM.2021.3114551</identifier><identifier>CODEN: ITMUF8</identifier><language>eng</language><publisher>Piscataway: IEEE</publisher><subject>Artificial neural networks ; authentic distortion ; convolutional neural network ; Convolutional neural networks ; Datasets ; Deep learning ; Distortion ; Feature extraction ; Heuristic methods ; Image quality ; Lower bounds ; No-reference image quality assessment ; Prediction models ; Predictive models ; Quality assessment ; rank learning ; Ranking ; Source code ; synthetic distortion ; Training ; Transform coding ; Weber-Fechner law</subject><ispartof>IEEE transactions on multimedia, 2022-01, Vol.24, p.4197-4211</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c206t-28de292e0f220105966cb3e0c678d202a8fe5ddfcf47ccda116188cfb88610d23</citedby><cites>FETCH-LOGICAL-c206t-28de292e0f220105966cb3e0c678d202a8fe5ddfcf47ccda116188cfb88610d23</cites><orcidid>0000-0003-0385-8793 ; 0000-0001-7484-7261 ; 0000-0003-3010-4196 ; 0000-0001-7956-5343</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/9548827$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,780,784,27924,27925,54796</link.rule.ids></links><search><creatorcontrib>Ou, Fu-Zhao</creatorcontrib><creatorcontrib>Wang, Yuan-Gen</creatorcontrib><creatorcontrib>Li, Jin</creatorcontrib><creatorcontrib>Zhu, Guopu</creatorcontrib><creatorcontrib>Kwong, Sam</creatorcontrib><title>A Novel Rank Learning Based No-Reference Image Quality Assessment Method</title><title>IEEE transactions on multimedia</title><addtitle>TMM</addtitle><description><![CDATA[Recently, applying deep learning to no-reference image quality assessment (NR-IQA) has received significant attention. Especially in the last five years, an increasing interest has been drawn to the studies of rank learning since it can help mitigate the problem of small IQA datasets. However, on one hand, existing rank learning is not suitable for the authentically distorted images due to the lack of generated rank samples. On the other hand, the output of existing rank loss functions is uncontrollable, resulting in reduced performance. Motivated by these two limitations, we propose a novel rank learning based NR-IQA method, termed controllable list-wise ranking IQA (CLRIQA) in this paper. To be specific, we first present an imaging-heuristic approach, in which the over- and under-exposure is formulated as an inverse of the Weber-Fechner law, and fusion strategy and compression are adopted, to simulate the authentic distortion and generate the rank image samples. These samples are label-free yet associated with quality ranking information. Then we design a controllable list-wise ranking (CLR) loss function by setting an upper and lower bound of rank range and introducing an adaptive margin to tune rank interval. Finally, both the generated rank samples and proposed CLR are used to pre-train a convolutional neural network. Moreover, to obtain a more accurate prediction model, we take advantage of the IQA datasets to fine-tune the pre-trained network further. Various experiments are conducted on the IQA benchmark datasets, and experimental results demonstrate the effectiveness of the proposed CLRIQA method. The source code and network model can be downloaded at the following web address: https://github.com <inline-formula><tex-math notation="LaTeX">/</tex-math></inline-formula> GZHU-DVL <inline-formula><tex-math notation="LaTeX">/</tex-math></inline-formula> CLRIQA .]]></description><subject>Artificial neural networks</subject><subject>authentic distortion</subject><subject>convolutional neural network</subject><subject>Convolutional neural networks</subject><subject>Datasets</subject><subject>Deep learning</subject><subject>Distortion</subject><subject>Feature extraction</subject><subject>Heuristic methods</subject><subject>Image quality</subject><subject>Lower bounds</subject><subject>No-reference image quality assessment</subject><subject>Prediction models</subject><subject>Predictive models</subject><subject>Quality assessment</subject><subject>rank learning</subject><subject>Ranking</subject><subject>Source code</subject><subject>synthetic distortion</subject><subject>Training</subject><subject>Transform coding</subject><subject>Weber-Fechner law</subject><issn>1520-9210</issn><issn>1941-0077</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><recordid>eNo9kNFLwzAQh4MoOKfvgi8Fnzvv0qZJH6eoG2yKYz6HLL3Mzq2dSSfsvzdjw6c7uO93d3yM3SIMEKF8mE-nAw4cBxliLgSesR6WOaYAUp7HXnBIS45wya5CWAFECGSPjYbJW_tL62Rmmu9kQsY3dbNMHk2gKk7SGTny1FhKxhuzpORjZ9Z1t0-GIVAIG2q6ZErdV1tdswtn1oFuTrXPPl-e50-jdPL-On4aTlLLoehSririJSdwnAOCKIvCLjICW0hVxf-NciSqylmXS2srg1igUtYtlCoQKp712f1x79a3PzsKnV61O9_Ek5pL5CCEkCpScKSsb0Pw5PTW1xvj9xpBH3zp6EsffOmTrxi5O0ZqIvrHS5ErxWX2ByCJZNA</recordid><startdate>20220101</startdate><enddate>20220101</enddate><creator>Ou, Fu-Zhao</creator><creator>Wang, Yuan-Gen</creator><creator>Li, Jin</creator><creator>Zhu, Guopu</creator><creator>Kwong, Sam</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><orcidid>https://orcid.org/0000-0003-0385-8793</orcidid><orcidid>https://orcid.org/0000-0001-7484-7261</orcidid><orcidid>https://orcid.org/0000-0003-3010-4196</orcidid><orcidid>https://orcid.org/0000-0001-7956-5343</orcidid></search><sort><creationdate>20220101</creationdate><title>A Novel Rank Learning Based No-Reference Image Quality Assessment Method</title><author>Ou, Fu-Zhao ; Wang, Yuan-Gen ; Li, Jin ; Zhu, Guopu ; Kwong, Sam</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c206t-28de292e0f220105966cb3e0c678d202a8fe5ddfcf47ccda116188cfb88610d23</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Artificial neural networks</topic><topic>authentic distortion</topic><topic>convolutional neural network</topic><topic>Convolutional neural networks</topic><topic>Datasets</topic><topic>Deep learning</topic><topic>Distortion</topic><topic>Feature extraction</topic><topic>Heuristic methods</topic><topic>Image quality</topic><topic>Lower bounds</topic><topic>No-reference image quality assessment</topic><topic>Prediction models</topic><topic>Predictive models</topic><topic>Quality assessment</topic><topic>rank learning</topic><topic>Ranking</topic><topic>Source code</topic><topic>synthetic distortion</topic><topic>Training</topic><topic>Transform coding</topic><topic>Weber-Fechner law</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Ou, Fu-Zhao</creatorcontrib><creatorcontrib>Wang, Yuan-Gen</creatorcontrib><creatorcontrib>Li, Jin</creatorcontrib><creatorcontrib>Zhu, Guopu</creatorcontrib><creatorcontrib>Kwong, Sam</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005–Present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>IEEE transactions on multimedia</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Ou, Fu-Zhao</au><au>Wang, Yuan-Gen</au><au>Li, Jin</au><au>Zhu, Guopu</au><au>Kwong, Sam</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A Novel Rank Learning Based No-Reference Image Quality Assessment Method</atitle><jtitle>IEEE transactions on multimedia</jtitle><stitle>TMM</stitle><date>2022-01-01</date><risdate>2022</risdate><volume>24</volume><spage>4197</spage><epage>4211</epage><pages>4197-4211</pages><issn>1520-9210</issn><eissn>1941-0077</eissn><coden>ITMUF8</coden><abstract><![CDATA[Recently, applying deep learning to no-reference image quality assessment (NR-IQA) has received significant attention. Especially in the last five years, an increasing interest has been drawn to the studies of rank learning since it can help mitigate the problem of small IQA datasets. However, on one hand, existing rank learning is not suitable for the authentically distorted images due to the lack of generated rank samples. On the other hand, the output of existing rank loss functions is uncontrollable, resulting in reduced performance. Motivated by these two limitations, we propose a novel rank learning based NR-IQA method, termed controllable list-wise ranking IQA (CLRIQA) in this paper. To be specific, we first present an imaging-heuristic approach, in which the over- and under-exposure is formulated as an inverse of the Weber-Fechner law, and fusion strategy and compression are adopted, to simulate the authentic distortion and generate the rank image samples. These samples are label-free yet associated with quality ranking information. Then we design a controllable list-wise ranking (CLR) loss function by setting an upper and lower bound of rank range and introducing an adaptive margin to tune rank interval. Finally, both the generated rank samples and proposed CLR are used to pre-train a convolutional neural network. Moreover, to obtain a more accurate prediction model, we take advantage of the IQA datasets to fine-tune the pre-trained network further. Various experiments are conducted on the IQA benchmark datasets, and experimental results demonstrate the effectiveness of the proposed CLRIQA method. The source code and network model can be downloaded at the following web address: https://github.com <inline-formula><tex-math notation="LaTeX">/</tex-math></inline-formula> GZHU-DVL <inline-formula><tex-math notation="LaTeX">/</tex-math></inline-formula> CLRIQA .]]></abstract><cop>Piscataway</cop><pub>IEEE</pub><doi>10.1109/TMM.2021.3114551</doi><tpages>15</tpages><orcidid>https://orcid.org/0000-0003-0385-8793</orcidid><orcidid>https://orcid.org/0000-0001-7484-7261</orcidid><orcidid>https://orcid.org/0000-0003-3010-4196</orcidid><orcidid>https://orcid.org/0000-0001-7956-5343</orcidid></addata></record> |
fulltext | fulltext |
identifier | ISSN: 1520-9210 |
ispartof | IEEE transactions on multimedia, 2022-01, Vol.24, p.4197-4211 |
issn | 1520-9210 1941-0077 |
language | eng |
recordid | cdi_proquest_journals_2712055578 |
source | IEEE Electronic Library (IEL) Journals |
subjects | Artificial neural networks authentic distortion convolutional neural network Convolutional neural networks Datasets Deep learning Distortion Feature extraction Heuristic methods Image quality Lower bounds No-reference image quality assessment Prediction models Predictive models Quality assessment rank learning Ranking Source code synthetic distortion Training Transform coding Weber-Fechner law |
title | A Novel Rank Learning Based No-Reference Image Quality Assessment Method |
url | http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-24T20%3A54%3A51IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20Novel%20Rank%20Learning%20Based%20No-Reference%20Image%20Quality%20Assessment%20Method&rft.jtitle=IEEE%20transactions%20on%20multimedia&rft.au=Ou,%20Fu-Zhao&rft.date=2022-01-01&rft.volume=24&rft.spage=4197&rft.epage=4211&rft.pages=4197-4211&rft.issn=1520-9210&rft.eissn=1941-0077&rft.coden=ITMUF8&rft_id=info:doi/10.1109/TMM.2021.3114551&rft_dat=%3Cproquest_cross%3E2712055578%3C/proquest_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c206t-28de292e0f220105966cb3e0c678d202a8fe5ddfcf47ccda116188cfb88610d23%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2712055578&rft_id=info:pmid/&rft_ieee_id=9548827&rfr_iscdi=true |