Consistent Variable Selection for a Nonparametric Nonlinear System by Inverse and Contour Regressions

• Published in: IEEE transactions on automatic control 2019-07, Vol.64 (7), p.2653-2664
• Main Authors: Cheng, Changming, Bai, Er-wei, Peng, Zhike
• Format: Article
• Language: English
• Subjects: Algorithms; Contours; Data models; Dimensionality reduction; Estimation; Independent variables; Input variables; Kernel; Mathematical models; Mutual information; nonlinear identification; Nonlinear systems; Nonparametric statistics; nonparametric systems; Regression analysis; Shape; variable selection
• ISSN: 0018-9286; 1558-2523
• DOI: 10.1109/TAC.2018.2867252

A parsimonious model is always preferred in engineering applications not only because it has a better prediction ability but also because it suffers less from the curse of dimensionality in data-based modeling. One way to achieve a parsimonious model is to identify contributing variables from the candidate variables and then to eliminate noncontributing or redundant variables. However, identifying which variables contribute and which variables do not contribute is not an easy task for a nonparametric nonlinear system. This paper considers variable-selection problems for a nonlinear nonparametric system. Two approaches, inverse and contour variable-selection algorithms, are proposed along with their theoretical analysis and numerical algorithms. Neither approach suffers from the curse of dimensionality, which is usually a problem for traditional variable-selection methods for a nonparametric nonlinear system. Furthermore, no elliptic symmetry nor independent input variables are assumed, so both algorithms enjoy wide applications. Numerical algorithms for both approaches are fairly straightforward and simple.