Wide spectrum feature selection (WiSe) for regression model building

• Published in: Computers & chemical engineering 2019-02, Vol.121, p.99-110
• Main Authors: Rendall, Ricardo, Castillo, Ivan, Schmidt, Alix, Chin, Swee-Teng, Chiang, Leo H., Reis, Marco
• Format: Article
• Language: English
• Subjects: Effect sparsity; Feature selection; Filtering methods; Predictive analytics; Symmetrical uncertainty
• ISSN: 0098-1354; 1873-4375
• DOI: 10.1016/j.compchemeng.2018.10.005

•A new two-stage feature selection methodology for high-dimensional regression problems is proposed.•It is called wide spectrum feature selection for regression (WiSe).•Pearson correlation, Spearman rank correlation, Symmetrical uncertainty, and all their pairwise combinations are considered in the first stage.•In the second stage, the first-pass screened features are further selected using regression-dependent approaches.•Results confirm the increased predictive accuracy of models built after filtering out irrelevant features. Developing predictive models from industrial datasets implies the consideration of many possible predictor variables (features). Using all available features for data-driven modelling is not recommended, as most of them are expected to be irrelevant and their inclusion in the model may compromise robustness and accuracy. In this work, we present, test and compare a new two-stage feature selection method called wide spectrum feature selection for regression (WiSe). In the first stage, a combination of efficient bivariate filters analyzes linear and non-linear association patterns between predictors and responses, screening out clearly noisy features. In the second stage, the reduced set of retained features is subject to further selection in the scope of the predictive methods considered, optimizing their predictive performance. Three simulated datasets and an industrial case illustrate the effectiveness and benefits of applying WiSe to support model development in a wide range of high-dimensional regression problems.