Loading…

A modified entropy-based performance criterion for class-modelling with multiple classes

The paper presents a new proposal for a single overall measure, the diagonal modified confusion entropy (DMCEN), to assess the performance of class-models jointly computed for several classes, a versatile index regarding sensitivity and specificity, and that supports class weighting. The characteris...

Full description

Saved in:
Bibliographic Details
Published in:Chemometrics and intelligent laboratory systems 2021-10, Vol.217, p.104423, Article 104423
Main Authors: Valencia, O., Ortiz, M.C., Sánchez, M.S., Sarabia, L.A.
Format: Article
Language:English
Subjects:
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c360t-8b64b04fb612abce1cba4ba8f9e9235e2ab69c902c8ba23f85fc1d885bcf8f483
cites cdi_FETCH-LOGICAL-c360t-8b64b04fb612abce1cba4ba8f9e9235e2ab69c902c8ba23f85fc1d885bcf8f483
container_end_page
container_issue
container_start_page 104423
container_title Chemometrics and intelligent laboratory systems
container_volume 217
creator Valencia, O.
Ortiz, M.C.
Sánchez, M.S.
Sarabia, L.A.
description The paper presents a new proposal for a single overall measure, the diagonal modified confusion entropy (DMCEN), to assess the performance of class-models jointly computed for several classes, a versatile index regarding sensitivity and specificity, and that supports class weighting. The characteristics of the proposed figure of merit are illustrated as against other usual performance measures and show how the index is more sensitive to the variations in the class-models than similar published indexes. Besides, a benchmark value representing a random modelling is also defined for DMCEN to be used as initial level to assess the quality of the built class-models. Furthermore, systematic comparisons have been conducted by using the degree of consistency C and the degree of discriminancy D when comparing the proposed DMCEN to the usual total efficiency (a geometric mean between sensitivity and specificity). Simulations show that, for a hundred thousand sensitivity/specificity matrices with four categories, C is almost 0.7 on average, well above the needed 0.5, and there is more than 62% probability that DMCEN detects differences when the total efficiency does not. Illustration of the application of the index is shown with an experimental data set with four categories. •Proposal of a new more sensitive performance measure, DMCEN, for several class-models.•An entropy-based, versatile enough index that detect changes in sensitivity and specificity.•DMCEN outperforms the total efficiency in both consistency and discriminancy criteria.•A benchmark, threshold value, for DMCEN allows discarding random class-models.
doi_str_mv 10.1016/j.chemolab.2021.104423
format article
fullrecord <record><control><sourceid>elsevier_cross</sourceid><recordid>TN_cdi_crossref_primary_10_1016_j_chemolab_2021_104423</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S016974392100191X</els_id><sourcerecordid>S016974392100191X</sourcerecordid><originalsourceid>FETCH-LOGICAL-c360t-8b64b04fb612abce1cba4ba8f9e9235e2ab69c902c8ba23f85fc1d885bcf8f483</originalsourceid><addsrcrecordid>eNqFkN1KAzEQhYMoWKuvIHmB1Px1m72zFLVCwRsF70KSndiU3c2SrErf3pTVa6-Gc2bOMPMhdMvoglFW3R0Wbg9dbI1dcMpZMaXk4gzNmFoJIrioz9GsDNZkJUV9ia5yPtCTlmyG3te4i03wARoM_ZjicCTW5KIGSD6mzvQOsEthhBRij4uFXWtyJiUGbRv6D_wdxj3uPtsxDC1MXcjX6MKbNsPNb52jt8eH182W7F6enjfrHXGioiNRtpKWSm8rxo11wJw10hrla6i5WEIxq9rVlDtlDRdeLb1jjVJL67zyUok5qqa9LsWcE3g9pNCZdNSM6hMffdB_fPSJj574lOD9FIRy3VeApLMLUL5tQgI36iaG_1b8AG9udKU</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>A modified entropy-based performance criterion for class-modelling with multiple classes</title><source>Elsevier</source><creator>Valencia, O. ; Ortiz, M.C. ; Sánchez, M.S. ; Sarabia, L.A.</creator><creatorcontrib>Valencia, O. ; Ortiz, M.C. ; Sánchez, M.S. ; Sarabia, L.A.</creatorcontrib><description>The paper presents a new proposal for a single overall measure, the diagonal modified confusion entropy (DMCEN), to assess the performance of class-models jointly computed for several classes, a versatile index regarding sensitivity and specificity, and that supports class weighting. The characteristics of the proposed figure of merit are illustrated as against other usual performance measures and show how the index is more sensitive to the variations in the class-models than similar published indexes. Besides, a benchmark value representing a random modelling is also defined for DMCEN to be used as initial level to assess the quality of the built class-models. Furthermore, systematic comparisons have been conducted by using the degree of consistency C and the degree of discriminancy D when comparing the proposed DMCEN to the usual total efficiency (a geometric mean between sensitivity and specificity). Simulations show that, for a hundred thousand sensitivity/specificity matrices with four categories, C is almost 0.7 on average, well above the needed 0.5, and there is more than 62% probability that DMCEN detects differences when the total efficiency does not. Illustration of the application of the index is shown with an experimental data set with four categories. •Proposal of a new more sensitive performance measure, DMCEN, for several class-models.•An entropy-based, versatile enough index that detect changes in sensitivity and specificity.•DMCEN outperforms the total efficiency in both consistency and discriminancy criteria.•A benchmark, threshold value, for DMCEN allows discarding random class-models.</description><identifier>ISSN: 0169-7439</identifier><identifier>EISSN: 1873-3239</identifier><identifier>DOI: 10.1016/j.chemolab.2021.104423</identifier><language>eng</language><publisher>Elsevier B.V</publisher><subject>Benchmark ; Class-model ; Entropy ; Sensitivity ; Specificity ; Type I and Type II errors</subject><ispartof>Chemometrics and intelligent laboratory systems, 2021-10, Vol.217, p.104423, Article 104423</ispartof><rights>2021 The Author(s)</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c360t-8b64b04fb612abce1cba4ba8f9e9235e2ab69c902c8ba23f85fc1d885bcf8f483</citedby><cites>FETCH-LOGICAL-c360t-8b64b04fb612abce1cba4ba8f9e9235e2ab69c902c8ba23f85fc1d885bcf8f483</cites><orcidid>0000-0002-4751-8929 ; 0000-0002-5936-7929</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,776,780,27901,27902</link.rule.ids></links><search><creatorcontrib>Valencia, O.</creatorcontrib><creatorcontrib>Ortiz, M.C.</creatorcontrib><creatorcontrib>Sánchez, M.S.</creatorcontrib><creatorcontrib>Sarabia, L.A.</creatorcontrib><title>A modified entropy-based performance criterion for class-modelling with multiple classes</title><title>Chemometrics and intelligent laboratory systems</title><description>The paper presents a new proposal for a single overall measure, the diagonal modified confusion entropy (DMCEN), to assess the performance of class-models jointly computed for several classes, a versatile index regarding sensitivity and specificity, and that supports class weighting. The characteristics of the proposed figure of merit are illustrated as against other usual performance measures and show how the index is more sensitive to the variations in the class-models than similar published indexes. Besides, a benchmark value representing a random modelling is also defined for DMCEN to be used as initial level to assess the quality of the built class-models. Furthermore, systematic comparisons have been conducted by using the degree of consistency C and the degree of discriminancy D when comparing the proposed DMCEN to the usual total efficiency (a geometric mean between sensitivity and specificity). Simulations show that, for a hundred thousand sensitivity/specificity matrices with four categories, C is almost 0.7 on average, well above the needed 0.5, and there is more than 62% probability that DMCEN detects differences when the total efficiency does not. Illustration of the application of the index is shown with an experimental data set with four categories. •Proposal of a new more sensitive performance measure, DMCEN, for several class-models.•An entropy-based, versatile enough index that detect changes in sensitivity and specificity.•DMCEN outperforms the total efficiency in both consistency and discriminancy criteria.•A benchmark, threshold value, for DMCEN allows discarding random class-models.</description><subject>Benchmark</subject><subject>Class-model</subject><subject>Entropy</subject><subject>Sensitivity</subject><subject>Specificity</subject><subject>Type I and Type II errors</subject><issn>0169-7439</issn><issn>1873-3239</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><recordid>eNqFkN1KAzEQhYMoWKuvIHmB1Px1m72zFLVCwRsF70KSndiU3c2SrErf3pTVa6-Gc2bOMPMhdMvoglFW3R0Wbg9dbI1dcMpZMaXk4gzNmFoJIrioz9GsDNZkJUV9ia5yPtCTlmyG3te4i03wARoM_ZjicCTW5KIGSD6mzvQOsEthhBRij4uFXWtyJiUGbRv6D_wdxj3uPtsxDC1MXcjX6MKbNsPNb52jt8eH182W7F6enjfrHXGioiNRtpKWSm8rxo11wJw10hrla6i5WEIxq9rVlDtlDRdeLb1jjVJL67zyUok5qqa9LsWcE3g9pNCZdNSM6hMffdB_fPSJj574lOD9FIRy3VeApLMLUL5tQgI36iaG_1b8AG9udKU</recordid><startdate>20211015</startdate><enddate>20211015</enddate><creator>Valencia, O.</creator><creator>Ortiz, M.C.</creator><creator>Sánchez, M.S.</creator><creator>Sarabia, L.A.</creator><general>Elsevier B.V</general><scope>6I.</scope><scope>AAFTH</scope><scope>AAYXX</scope><scope>CITATION</scope><orcidid>https://orcid.org/0000-0002-4751-8929</orcidid><orcidid>https://orcid.org/0000-0002-5936-7929</orcidid></search><sort><creationdate>20211015</creationdate><title>A modified entropy-based performance criterion for class-modelling with multiple classes</title><author>Valencia, O. ; Ortiz, M.C. ; Sánchez, M.S. ; Sarabia, L.A.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c360t-8b64b04fb612abce1cba4ba8f9e9235e2ab69c902c8ba23f85fc1d885bcf8f483</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Benchmark</topic><topic>Class-model</topic><topic>Entropy</topic><topic>Sensitivity</topic><topic>Specificity</topic><topic>Type I and Type II errors</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Valencia, O.</creatorcontrib><creatorcontrib>Ortiz, M.C.</creatorcontrib><creatorcontrib>Sánchez, M.S.</creatorcontrib><creatorcontrib>Sarabia, L.A.</creatorcontrib><collection>ScienceDirect Open Access Titles</collection><collection>Elsevier:ScienceDirect:Open Access</collection><collection>CrossRef</collection><jtitle>Chemometrics and intelligent laboratory systems</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Valencia, O.</au><au>Ortiz, M.C.</au><au>Sánchez, M.S.</au><au>Sarabia, L.A.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A modified entropy-based performance criterion for class-modelling with multiple classes</atitle><jtitle>Chemometrics and intelligent laboratory systems</jtitle><date>2021-10-15</date><risdate>2021</risdate><volume>217</volume><spage>104423</spage><pages>104423-</pages><artnum>104423</artnum><issn>0169-7439</issn><eissn>1873-3239</eissn><abstract>The paper presents a new proposal for a single overall measure, the diagonal modified confusion entropy (DMCEN), to assess the performance of class-models jointly computed for several classes, a versatile index regarding sensitivity and specificity, and that supports class weighting. The characteristics of the proposed figure of merit are illustrated as against other usual performance measures and show how the index is more sensitive to the variations in the class-models than similar published indexes. Besides, a benchmark value representing a random modelling is also defined for DMCEN to be used as initial level to assess the quality of the built class-models. Furthermore, systematic comparisons have been conducted by using the degree of consistency C and the degree of discriminancy D when comparing the proposed DMCEN to the usual total efficiency (a geometric mean between sensitivity and specificity). Simulations show that, for a hundred thousand sensitivity/specificity matrices with four categories, C is almost 0.7 on average, well above the needed 0.5, and there is more than 62% probability that DMCEN detects differences when the total efficiency does not. Illustration of the application of the index is shown with an experimental data set with four categories. •Proposal of a new more sensitive performance measure, DMCEN, for several class-models.•An entropy-based, versatile enough index that detect changes in sensitivity and specificity.•DMCEN outperforms the total efficiency in both consistency and discriminancy criteria.•A benchmark, threshold value, for DMCEN allows discarding random class-models.</abstract><pub>Elsevier B.V</pub><doi>10.1016/j.chemolab.2021.104423</doi><orcidid>https://orcid.org/0000-0002-4751-8929</orcidid><orcidid>https://orcid.org/0000-0002-5936-7929</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 0169-7439
ispartof Chemometrics and intelligent laboratory systems, 2021-10, Vol.217, p.104423, Article 104423
issn 0169-7439
1873-3239
language eng
recordid cdi_crossref_primary_10_1016_j_chemolab_2021_104423
source Elsevier
subjects Benchmark
Class-model
Entropy
Sensitivity
Specificity
Type I and Type II errors
title A modified entropy-based performance criterion for class-modelling with multiple classes
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-30T04%3A23%3A04IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-elsevier_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20modified%20entropy-based%20performance%20criterion%20for%20class-modelling%20with%20multiple%20classes&rft.jtitle=Chemometrics%20and%20intelligent%20laboratory%20systems&rft.au=Valencia,%20O.&rft.date=2021-10-15&rft.volume=217&rft.spage=104423&rft.pages=104423-&rft.artnum=104423&rft.issn=0169-7439&rft.eissn=1873-3239&rft_id=info:doi/10.1016/j.chemolab.2021.104423&rft_dat=%3Celsevier_cross%3ES016974392100191X%3C/elsevier_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c360t-8b64b04fb612abce1cba4ba8f9e9235e2ab69c902c8ba23f85fc1d885bcf8f483%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true