Using Autoencoders for Anomaly Detection in Hard Disk Drives

• Main Authors: Pereira, Francisco Lucas F., Castro Chaves, Iago, Gomes, Joao Paulo P., Machado, Javam C.
• Format: Conference Proceeding
• Language: English
• Subjects: Anomaly detection; autoencoder; Decoding; Fault detection; hard disk drives; Hard disks; Machine learning; Measurement; Principal component analysis
• ISSN: 2161-4407
• DOI: 10.1109/IJCNN48605.2020.9206689

Nowadays, predicting failures in Hard Disk Drives (HDD) is of key importance for storage service providers and end users. Being able to detect in advance that a disk is going to fail may enable maintenance actions that can avoid severe data losses. For that reason, many researchers had devoted attention to this research topic. Recently, several authors have reported promising results by using attributes collected by the SMART (Self-Monitoring, Analysis and Reporting Technology) system along with machine learning methods. Although the best results were obtained by supervised machine learning methods, it is important to notice that data from degraded HDDs is scarce. Hence, anomaly detection methods arise a promising solution. Among such methods, recent works reported that reconstruction based anomaly detection algorithms had the best performance on HDDs fault detection. In line with such results, in this paper we aim to further investigate the performance of such methods. We conducted tests with classical PCA based methods and neural autoencoder based methods. In addition to testing with the popular reconstruction based autoencoder method we also evaluated a method that analyzes the distribution of the latent space. Additionally we propose a simple formulation to combine both methods. On the basis of our experiments, we verified that autoencoder based methods had the best performances according to the two evaluation metrics. Among such methods, the combination approach had the best overall performance.