Measurement Invariance Investigation for Performance of Deep Learning Architectures

Models can be compared against each other. Such comparison helps to find strengths and weaknesses of different approaches and gives additional possibilities for model validation. Multi-class classification is a classification task where each image is assigned to one and only one label. Confusion mat...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2022, Vol.10, p.78070-78087
Main Authors: Chen, Dewang, Lu, Yuqi, Hsu, Chih-Yu
Format: Article
Language:English
Subjects:
Accuracy
Business metrics
Convolutional neural networks
Correlation coefficients
Datasets
Deep learning
Deep learning model
Diseases
Image classification
Indicators
Invariance
Machine learning
measurement invariance
multivariate linear regression
Neural networks
Pearson correlation coefficient
performance indicators
Performance measurement
Predictive models
Regression analysis
Robustness (mathematics)
Training
Training data
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Models can be compared against each other. Such comparison helps to find strengths and weaknesses of different approaches and gives additional possibilities for model validation. Multi-class classification is a classification task where each image is assigned to one and only one label. Confusion matrix, Precision, Recall, and F1 Score are popular performance metrics. The common sense is that the performance of any architecture is dependent on the sizes of data set. The goodness of the architecture of deep learning models for different data sets is critical issue. This paper implements Pearson correlation coefficient and the multivariate linear regression method to assess the accuracy of deep learning architectures with five performance indicators. The five performance indicators are training loss rate, robustness, training time, number of model parameters and computation complexity. There are five image datasets used to test four deep learning models: Alexnet, GoogLeNet, ResNet, MobileNet to obtain the values of accuracy and other five indicators. The most important contribution of the article is to show that the accuracy indicator related to training loss rate and training time indicators are not dependent on the selection of the data group. According to the definition of Measurement Invariance (MI), the measurement invariance is demonstrated by the linear regression analysis and inner product of the unit normal vector of the linear regression planes.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2022.3192468