Tensor-factorized neural networks

The growing interests in multiway data analysis and deep learning have drawn tensor factorization (TF) and neural network (NN) as the crucial topics. Conventionally, the NN model is estimated from a set of one-way observations. Such a vectorized NN is not generalized for learning the representation...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transaction on neural networks and learning systems 2018-05, Vol.29 (5), p.1998-2011
Main Authors: Chien, Jen-Tzung, Bao, Yi-Ting
Format: Article
Language:English
Subjects:
Affine transformations
Artificial neural networks
Back propagation
Backpropagation
Classification
Convolution
Data analysis
Data processing
Data structures
Factorization
Feature extraction
Machine learning
Neural network (NN)
Neural networks
Parameters
pattern classification
Spatial data
Speech
Tensile stress
tensor factorization (TF)
tensor-factorized error backpropagation
Training
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The growing interests in multiway data analysis and deep learning have drawn tensor factorization (TF) and neural network (NN) as the crucial topics. Conventionally, the NN model is estimated from a set of one-way observations. Such a vectorized NN is not generalized for learning the representation from multiway observations. The classification performance using vectorized NN is constrained, because the temporal or spatial information in neighboring ways is disregarded. More parameters are required to learn the complicated data structure. This paper presents a new tensor-factorized NN (TFNN), which tightly integrates TF and NN for multiway feature extraction and classification under a unified discriminative objective. This TFNN is seen as a generalized NN, where the affine transformation in an NN is replaced by the multilinear and multiway factorization for tensor-based NN. The multiway information is preserved through layerwise factorization. Tucker decomposition and nonlinear activation are performed in each hidden layer. The tensor-factorized error backpropagation is developed to train TFNN with the limited parameter size and computation time. This TFNN can be further extended to realize the convolutional TFNN (CTFNN) by looking at small subtensors through the factorized convolution. Experiments on real-world classification tasks demonstrate that TFNN and CTFNN attain substantial improvement when compared with an NN and a convolutional NN, respectively.
ISSN:2162-237X
2162-2388
DOI:10.1109/TNNLS.2017.2690379