Biobjective gradient descent for feature selection on high dimension, low sample size data

Even though deep learning shows impressive results in several applications, its use on problems with High Dimensions and Low Sample Size, such as diagnosing rare diseases, leads to overfitting. One solution often proposed is feature selection. In deep learning, along with feature selection, network...

Full description

Saved in:
Bibliographic Details
Published in:PloS one 2024-07, Vol.19 (7), p.e0305654
Main Authors: Issa, Tina, Angel, Eric, Zehraoui, Farida
Format: Article
Language:English
Subjects:
Algorithms
Artificial Intelligence
Artificial neural networks
Bioinformatics
Biology and Life Sciences
Computer and Information Sciences
Computer Science
Constraints
Datasets
Deep Learning
Feature selection
Genetic algorithms
Humans
Machine learning
Methods
Neural networks
Neural Networks, Computer
Optimization
Physical Sciences
Research and Analysis Methods
Sample Size
Social networks
Sparsity
Statistical analysis
Statistical tests
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Even though deep learning shows impressive results in several applications, its use on problems with High Dimensions and Low Sample Size, such as diagnosing rare diseases, leads to overfitting. One solution often proposed is feature selection. In deep learning, along with feature selection, network sparsification is also used to improve the results when dealing with high dimensions low sample size data. However, most of the time, they are tackled as separate problems. This paper proposes a new approach that integrates feature selection, based on sparsification, into the training process of a deep neural network. This approach uses a constrained biobjective gradient descent method. It provides a set of Pareto optimal neural networks that make a trade-off between network sparsity and model accuracy. Results on both artificial and real datasets show that using a constrained biobjective gradient descent increases the network sparsity without degrading the classification performances. With the proposed approach, on an artificial dataset, the feature selection score reached 0.97 with a sparsity score of 0.92 with an accuracy of 0.9. For the same accuracy, none of the other methods reached a feature score above 0.20 and sparsity score of 0.35. Finally, statistical tests validate the results obtained on all datasets.
ISSN:1932-6203
1932-6203
DOI:10.1371/journal.pone.0305654