A novel multi-objective forest optimization algorithm for wrapper feature selection

• Published in: Expert systems with applications 2021-08, Vol.175, p.114737, Article 114737
• Main Authors: Nouri-Moghaddam, Babak, Ghazanfari, Mehdi, Fathian, Mohammad
• Format: Article
• Language: English
• Subjects: Algorithms; Classification; Datasets; Dimension reduction; Feature selection; Forest optimization algorithm; Multi-objective optimization; Multiple objective analysis; Optimization; Optimization algorithms; Representations; Wrapper method
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2021.114737

•A new multi-objective wrapper method based on Forest Optimization (MOFOA) is proposed.•MOFOA uses archive, grid, and region-based selection to maintain Pareto front.•Two continues and binary versions of MOFOA is presented to solve features selection.•Continuous version of MOFOA outperforms other multi-objective algorithms.•The performance of MOFOA was confirmed by quantitative and qualitative analyses. Feature selection is one of the important techniques of dimensionality reduction in data preprocessing because datasets generally have redundant and irrelevant features that adversely affect the performance and complexity of classification models. Feature selection has two main objectives, i.e., reducing the number of features and increasing classification performance due to its inherent nature. In this paper, we propose a multi-objective feature selection algorithm based on forest optimization algorithm (FOA) using the archive, grid, and region-based selection concepts. For this purpose, two versions of the proposed algorithm are developed using continuous and binary representations. The performance of the proposed algorithms is investigated on nine UCI datasets and two microarray datasets. Next, the obtained results are compared with seven traditional single-objective and five multi-objective methods. Based on the results, both proposed algorithms have reached the same performance or even outperformed the single-objective methods. Compared with other multi-objective algorithms, MOFOA with continuous representation has managed to reduce the classification error in most cases by selecting less number of features than other methods.