Tracing outliers in the dataset of Drosophila suzukii records with the Isolation Forest method

• Published in: Journal of big data 2020-03, Vol.7 (1), p.1-11, Article 14
• Main Authors: Santosuosso, Ugo, Cini, Alessandro, Papini, Alessio
• Format: Article
• Language: English
• Subjects: Algorithms; Anomalies; Big Data; Cluster analysis; Clustering; Communications Engineering; Computational Science and Engineering; Computer Science; Data analysis; Data Mining and Knowledge Discovery; Database Management; Datasets; Drosophila suzuki; Fluxes; Fruit flies; Geoprofiling; Information Storage and Retrieval; Insects; Invasion; Invasive species; Isolation Forest; K-Means; Machine learning; Mathematical Applications in Computer Science; Networks; Nonnative species; Outliers (statistics); Programming languages; Road transportation; Spatial data; Tessellation; Test procedures; Vector quantization
• ISSN: 2196-1115; 2196-1115
• DOI: 10.1186/s40537-020-00288-8

The analysis of big data is a fundamental challenge for the current and future stream of data coming from many different sources. Geospatial data is one of the sources currently less investigated. A typical example of always increasing data set is that produced by the distribution data of invasive species on the concerned territories. The dataset of Drosophila suzuki invasion sites in Europe up to 2011 was used to test a possible method to pinpoint its outliers (anomalies). Our aim was to find a method of analysis that would be able to treat large amount of data in order to produce easily readable outputs to summarize and predict the status and, possibly, the future development of a biological invasion. To do that, we aimed to identify the so called anomalies of the dataset, identified with a Python script based on the machine learning algorithm “Isolation Forest”. We used also the K-Means clustering method to partition the dataset. In our test, based on a real dataset, the Silhouette method yielded a number of clusters of 10 as the best result. The clusters were drawn on the map with a Voronoi tessellation, showing that 8 clusters were centered on industrial harbours, while the last two were in the hinterland. This fact led us to guess that: (1) the main entrance mechanisms in Europe may be the wares import fluxes through ports, occurring apparently several times; (2) the spreading into the inland may be due to road transportation of wares; (3) the outliers (anomalies) found with the isolation forest method would identify individuals or populations that tend to detach from their original cluster and hence represent indications about the lines of further spreading of the invasion. This type of analysis aims hence to identify the future direction of an invasion, rather than the center of origin as in the case of geographic profiling. Isolation Forest provides therefore complimentary results with respect to PGP. The recent records of the invasive species, mainly localized close to the outliers position, are an indication that the isolation forest method can be considered predictive and proved to be a useful method to treat large datasets of geospatial data.