Spatial neighborhood based anomaly detection in sensor datasets

• Published in: Data mining and knowledge discovery 2010-03, Vol.20 (2), p.221-258
• Main Authors: Janeja, Vandana P., Adam, Nabil R., Atluri, Vijayalakshmi, Vaidya, Jaideep
• Format: Article
• Language: English
• Subjects: Artificial Intelligence; Chemistry and Earth Sciences; Computer Science; Data mining; Data Mining and Knowledge Discovery; Datasets; Information Storage and Retrieval; Neighborhoods; Physics; Sensors; Spatial data; Statistics for Engineering; Toxicity
• ISSN: 1384-5810; 1573-756X
• DOI: 10.1007/s10618-009-0147-0

Success of anomaly detection, similar to other spatial data mining techniques, relies on neighborhood definition. In this paper, we argue that the anomalous behavior of spatial objects in a neighborhood can be truly captured when both (a) spatial autocorrelation (similar behavior of nearby objects due to proximity) and (b) spatial heterogeneity (distinct behavior of nearby objects due to difference in the underlying processes in the region) are taken into consideration for the neighborhood definition. Our approach begins by generating micro neighborhoods around spatial objects encompassing all the information about a spatial object. We selectively merge these based on spatial relationships accounting for autocorrelation and inferential relationships accounting for heterogeneity, forming macro neighborhoods . In such neighborhoods, we then identify (i) spatio-temporal outliers , where individual sensor readings are anomalous, (ii) spatial outliers , where the entire sensor is an anomaly, and (iii) spatio-temporally coalesced outliers , where a group of spatio-temporal outliers in the macro neighborhood are separated by a small time lag indicating the traversal of the anomaly. We demonstrate the effectiveness of our approach in neighborhood formation and anomaly detection with experimental results in (i) water monitoring and (ii) highway traffic monitoring sensor datasets. We also compare the results of our approach with an existing approach for spatial anomaly detection.