High-performance IoT streaming data prediction system using Spark: a case study of air pollution

Internet-of-Things (IoT) devices are becoming prevalent, and some of them, such as sensors, generate continuous time-series data, i.e., streaming data. These IoT streaming data are one of Big Data sources, and they require careful consideration for efficient data processing and analysis. Deep learni...

Full description

Saved in:
Bibliographic Details
Published in:Neural computing & applications 2020-09, Vol.32 (17), p.13147-13154
Main Authors: Jin, Ho-Yong, Jung, Eun-Sung, Lee, Duckki
Format: Article
Language:English
Subjects:
Air pollution
Alliances
Artificial Intelligence
Case studies
Computational Biology/Bioinformatics
Computational Science and Engineering
Computer Science
Data analysis
Data Mining and Knowledge Discovery
Data processing
Deep learning
Digital media
Image Processing and Computer Vision
Internet of Things
Machine learning
Neural networks
Probability and Statistics in Computer Science
S.I. : Green and Human Information Technology 2019
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Internet-of-Things (IoT) devices are becoming prevalent, and some of them, such as sensors, generate continuous time-series data, i.e., streaming data. These IoT streaming data are one of Big Data sources, and they require careful consideration for efficient data processing and analysis. Deep learning is emerging as a solution to IoT streaming data analytics. However, there is a persistent problem in deep learning that it takes a long time to learn neural networks. In this paper, we propose a high-performance IoT streaming data prediction system to improve the learning speed and to predict in real time. We showed the efficacy of the system through a case study of air pollution. The experimental results show that the modified LSTM autoencoder model shows the best performance compared to a generic LSTM model. We noticed that achieving the best performance requires optimizing many parameters, including learning rate, epoch, memory cell size, input timestep size, and the number of features/predictors. In that regard, we show that the high-performance data learning/prediction frameworks (e.g., Spark, Dist-Keras, and Hadoop) are essential to rapidly fine-tune a model for training and testing before real deployment of the model as data accumulate.
ISSN:0941-0643
1433-3058
DOI:10.1007/s00521-019-04678-9