Least Square Regularized Regression in Sum Space

This paper proposes a least square regularized regression algorithm in sum space of reproducing kernel Hilbert spaces (RKHSs) for nonflat function approximation, and obtains the solution of the algorithm by solving a system of linear equations. This algorithm can approximate the low- and high-freque...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transaction on neural networks and learning systems 2013-04, Vol.24 (4), p.635-646
Main Authors: XU, Yong-Li, CHEN, Di-Rong, LI, Han-Xiong, LU LIU
Format: Article
Language:English
Subjects:
Algorithms
Applied sciences
Approximation algorithms
Artificial intelligence
Computer science
control theory
systems
Computer simulation
Connectionism. Neural networks
Convergence
Covering
Data processing. List processing. Character string processing
Errors
Exact sciences and technology
Function approximation
Kernel
Kernels
Learning
Learning curves
Learning rate
least square regularized regression (LSRR)
Least squares approximation
Least squares method
Memory organisation. Data processing
multiscale kernel
Neural networks
Regression
reproducing kernel Hilbert space (RKHS)
Software
Studies
sum space
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper proposes a least square regularized regression algorithm in sum space of reproducing kernel Hilbert spaces (RKHSs) for nonflat function approximation, and obtains the solution of the algorithm by solving a system of linear equations. This algorithm can approximate the low- and high-frequency component of the target function with large and small scale kernels, respectively. The convergence and learning rate are analyzed. We measure the complexity of the sum space by its covering number and demonstrate that the covering number can be bounded by the product of the covering numbers of basic RKHSs. For sum space of RKHSs with Gaussian kernels, by choosing appropriate parameters, we tradeoff the sample error and regularization error, and obtain a polynomial learning rate, which is better than that in any single RKHS. The utility of this method is illustrated with two simulated data sets and five real-life databases.
ISSN:2162-237X
2162-2388
DOI:10.1109/TNNLS.2013.2242091