Ensembles of label noise filters: a ranking approach

• Published in: Data mining and knowledge discovery 2016-09, Vol.30 (5), p.1192-1216
• Main Authors: Garcia, Luís P. F., Lorena, Ana C., Matwin, Stan, de Carvalho, André C. P. L. F.
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial Intelligence; Bias; Case studies; Chemistry and Earth Sciences; Classification; Computational efficiency; Computer Science; Data mining; Data Mining and Knowledge Discovery; Datasets; Filtering; Filtration; Information Storage and Retrieval; Labels; Machine learning; Noise; Physics; Recommender systems; Statistics for Engineering; Tasks
• ISSN: 1384-5810; 1573-756X
• DOI: 10.1007/s10618-016-0475-9

Label noise can be a major problem in classification tasks, since most machine learning algorithms rely on data labels in their inductive process. Thereupon, various techniques for label noise identification have been investigated in the literature. The bias of each technique defines how suitable it is for each dataset. Besides, while some techniques identify a large number of examples as noisy and have a high false positive rate, others are very restrictive and therefore not able to identify all noisy examples. This paper investigates how label noise detection can be improved by using an ensemble of noise filtering techniques. These filters, individual and ensembles, are experimentally compared. Another concern in this paper is the computational cost of ensembles, once, for a particular dataset, an individual technique can have the same predictive performance as an ensemble. In this case the individual technique should be preferred. To deal with this situation, this study also proposes the use of meta-learning to recommend, for a new dataset, the best filter. An extensive experimental evaluation of the use of individual filters, ensemble filters and meta-learning was performed using public datasets with imputed label noise. The results show that ensembles of noise filters can improve noise filtering performance and that a recommendation system based on meta-learning can successfully recommend the best filtering technique for new datasets. A case study using a real dataset from the ecological niche modeling domain is also presented and evaluated, with the results validated by an expert.