A Multiobjective Sparse Feature Learning Model for Deep Neural Networks

Hierarchical deep neural networks are currently popular learning models for imitating the hierarchical architecture of human brain. Single-layer feature extractors are the bricks to build deep networks. Sparse feature learning models are popular models that can learn useful representations. But most...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transaction on neural networks and learning systems 2015-12, Vol.26 (12), p.3263-3277
Main Authors: Gong, Maoguo, Liu, Jia, Li, Hao, Cai, Qing, Su, Linzhi
Format: Article
Language:English
Subjects:
Algorithms
Artificial Intelligence
Brain modeling
Computer Simulation
Deep neural networks
evolutionary algorithm
Evolutionary computation
Feature extraction
Humans
Learning - physiology
Linear programming
Models, Theoretical
multiobjective optimization
Neural networks
Neural Networks (Computer)
Pareto optimization
sparse feature learning
Sparsity
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Hierarchical deep neural networks are currently popular learning models for imitating the hierarchical architecture of human brain. Single-layer feature extractors are the bricks to build deep networks. Sparse feature learning models are popular models that can learn useful representations. But most of those models need a user-defined constant to control the sparsity of representations. In this paper, we propose a multiobjective sparse feature learning model based on the autoencoder. The parameters of the model are learnt by optimizing two objectives, reconstruction error and the sparsity of hidden units simultaneously to find a reasonable compromise between them automatically. We design a multiobjective induced learning procedure for this model based on a multiobjective evolutionary algorithm. In the experiments, we demonstrate that the learning procedure is effective, and the proposed multiobjective model can learn useful sparse features.
ISSN:2162-237X
2162-2388
DOI:10.1109/TNNLS.2015.2469673