A modified entropy-based performance criterion for class-modelling with multiple classes

The paper presents a new proposal for a single overall measure, the diagonal modified confusion entropy (DMCEN), to assess the performance of class-models jointly computed for several classes, a versatile index regarding sensitivity and specificity, and that supports class weighting. The characteris...

Full description

Saved in:
Bibliographic Details
Published in:Chemometrics and intelligent laboratory systems 2021-10, Vol.217, p.104423, Article 104423
Main Authors: Valencia, O., Ortiz, M.C., Sánchez, M.S., Sarabia, L.A.
Format: Article
Language:English
Subjects:
Benchmark
Class-model
Entropy
Sensitivity
Specificity
Type I and Type II errors
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The paper presents a new proposal for a single overall measure, the diagonal modified confusion entropy (DMCEN), to assess the performance of class-models jointly computed for several classes, a versatile index regarding sensitivity and specificity, and that supports class weighting. The characteristics of the proposed figure of merit are illustrated as against other usual performance measures and show how the index is more sensitive to the variations in the class-models than similar published indexes. Besides, a benchmark value representing a random modelling is also defined for DMCEN to be used as initial level to assess the quality of the built class-models. Furthermore, systematic comparisons have been conducted by using the degree of consistency C and the degree of discriminancy D when comparing the proposed DMCEN to the usual total efficiency (a geometric mean between sensitivity and specificity). Simulations show that, for a hundred thousand sensitivity/specificity matrices with four categories, C is almost 0.7 on average, well above the needed 0.5, and there is more than 62% probability that DMCEN detects differences when the total efficiency does not. Illustration of the application of the index is shown with an experimental data set with four categories. •Proposal of a new more sensitive performance measure, DMCEN, for several class-models.•An entropy-based, versatile enough index that detect changes in sensitivity and specificity.•DMCEN outperforms the total efficiency in both consistency and discriminancy criteria.•A benchmark, threshold value, for DMCEN allows discarding random class-models.
ISSN:0169-7439
1873-3239
DOI:10.1016/j.chemolab.2021.104423