cleanTS: Automated (AutoML) tool to clean univariate time series at microscales

• Published in: Neurocomputing (Amsterdam) 2022-08, Vol.500, p.155-176
• Main Authors: Shende, Mayur Kishor, Feijóo-Lorenzo, Andrés E., Bokde, Neeraj Dhanraj
• Format: Article
• Language: English
• Subjects: AutoML; Data cleaning; Machine learning; Time series analysis; Time series cleaning
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2022.05.057

Data cleaning is one of the most important tasks in data analysis processes. One of the perennial challenges in data analytics is the detection and handling of non-valid data. Failing to do so can result in creating imbalanced observations that can cause bias and influence estimates, and in extreme cases, can even lead to inaccurate analytics and unreliable decisions. Usually, the process of data cleaning is time-consuming due to its growing volume, velocity, and variety. Further, the complexity and difficulty of the cleaning process increase with the amount of data to be analyzed. It is rarely the case that any real-world data is clean and error-free. Thus, pre-processing the data before using it for analysis has become standard practice. This paper is intended to provide an easy-to-use and reliable system which automates the cleaning process for univariate time series data. Also, automating the process reduces the time required for cleaning it. Another issue that the proposed system aims to solve is making the visualization of a large amount of data more effective. To tackle these issues, an R package, cleanTS is proposed. The proposed system provides a way to analyze data on different scales and resolutions. Also, it provides users with tools and a benchmark system for comparing various techniques used in data cleaning.