Feature Selection Using a Neural Network With Group Lasso Regularization and Controlled Redundancy

We propose a neural network-based feature selection (FS) scheme that can control the level of redundancy in the selected features by integrating two penalties into a single objective function. The Group Lasso penalty aims to produce sparsity in features in a grouped manner. The redundancy-control pe...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transaction on neural networks and learning systems 2021-03, Vol.32 (3), p.1110-1123
Main Authors: Wang, Jian, Zhang, Huaqing, Wang, Junze, Pu, Yifei, Pal, Nikhil R.
Format: Article
Language:English
Subjects:
Algorithms
Computational modeling
Control redundancy
Convergence
Correlation
Feature selection
feature selection (FS)
Fines & penalties
Group Lasso
Learning systems
monotonicity
Neural networks
Objective function
Redundancy
Regularization
Robustness
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose a neural network-based feature selection (FS) scheme that can control the level of redundancy in the selected features by integrating two penalties into a single objective function. The Group Lasso penalty aims to produce sparsity in features in a grouped manner. The redundancy-control penalty, which is defined based on a measure of dependence among features, is utilized to control the level of redundancy among the selected features. Both the penalty terms involve the L_{2,1} -norm of weight matrix between the input and hidden layers. These penalty terms are nonsmooth at the origin, and hence, one simple but efficient smoothing technique is employed to overcome this issue. The monotonicity and convergence of the proposed algorithm are specified and proved under suitable assumptions. Then, extensive experiments are conducted on both artificial and real data sets. Empirical results explicitly demonstrate the ability of the proposed FS scheme and its effectiveness in controlling redundancy. The empirical simulations are observed to be consistent with the theoretical results.
ISSN:2162-237X
2162-2388
DOI:10.1109/TNNLS.2020.2980383