A Backward Feature Selection by Creating Compact Neural Network Using Coherence Learning and Pruning

• Published in: Journal of advanced computational intelligence and intelligent informatics 2007-07, Vol.11 (6), p.570-581
• Main Authors: Kabir, Md. Monirul, Shahjahan, Md, Murase, Kazuyuki
• Format: Article
• Language: English
• ISSN: 1343-0130; 1883-8014
• DOI: 10.20965/jaciii.2007.p0570

In this paper we propose a new backward feature selection method that generates compact classifier of a three-layered feed-forward artificial neural network (ANN). In the algorithm, that is based on the wrapper model, two techniques, coherence and pruning, are integrated together in order to find relevant features with a network of minimal numbers of hidden units and connections. Firstly, a coherence learning and a pruning technique are applied during training for removing unnecessary hidden units from the network. After that, attribute distances are measured by a straightforward computation that is not computationally expensive. An attribute is then removed based on an error-based criterion. The network is retrained after the removal of the attribute. This unnecessary attribute selection process is continued until a stopping criterion is satisfied. We applied this method to several standard benchmark classification problems such as breast cancer, diabetes, glass identification and thyroid problems. Experimental results confirmed that the proposed method generates compact network structures that can select relevant features with good classification accuracies.