Unsupervised Anomaly Detection Based on Deep Autoencoding and Clustering

The unsupervised anomaly detection task based on high-dimensional or multidimensional data occupies a very important position in the field of machine learning and industrial applications; especially in the aspect of network security, the anomaly detection of network data is particularly important. T...

Full description

Saved in:
Bibliographic Details
Published in:Security and communication networks 2021, Vol.2021, p.1-8
Main Authors: Zhang, Chuanlei, Liu, Jiangtao, Chen, Wei, Shi, Jinyuan, Yao, Minda, Yan, Xiaoning, Xu, Nenghua, Chen, Dufeng
Format: Article
Language:English
Subjects:
Algorithms
Anomalies
Clustering
Datasets
Density
Discriminant analysis
Fault diagnosis
Industrial applications
Machine learning
Methods
Multidimensional data
Neural networks
Performance enhancement
Principal components analysis
Reduction
Representations
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The unsupervised anomaly detection task based on high-dimensional or multidimensional data occupies a very important position in the field of machine learning and industrial applications; especially in the aspect of network security, the anomaly detection of network data is particularly important. The key to anomaly detection is density estimation. Although the methods of dimension reduction and density estimation have made great progress in recent years, most dimension reduction methods are difficult to retain the key information of original data or multidimensional data. Recent studies have shown that the deep autoencoder (DAE) can solve this problem well. In order to improve the performance of unsupervised anomaly detection, we propose an anomaly detection scheme based on a deep autoencoder (DAE) and clustering methods. The deep autoencoder is trained to learn the compressed representation of the input data and then feed it to clustering approach. This scheme makes full use of the advantages of the deep autoencoder (DAE) to generate low-dimensional representation and reconstruction errors for the input high-dimensional or multidimensional data and uses them to reconstruct the input samples. The proposed scheme could eliminate redundant information contained in the data, improve performance of clustering methods in identifying abnormal samples, and reduce the amount of calculation. To verify the effectiveness of the proposed scheme, massive comparison experiments have been conducted with traditional dimension reduction algorithms and clustering methods. The results of experiments demonstrate that, in most cases, the proposed scheme outperforms the traditional dimension reduction algorithms with different clustering methods.
ISSN:1939-0114
1939-0122
DOI:10.1155/2021/7389943