Sensor data management in the cloud: Data storage, data ingestion, and data retrieval

Summary Sensors are widely used in the field of manufacturing, railways, aerospace, cars, medicines, robotics, and many other aspects of our everyday life. There is an increasing need to capture, store, and analyse the dynamic semi‐structured data from those sensors. A similar growth of semi‐structu...

Full description

Saved in:
Bibliographic Details
Published in:Concurrency and computation 2018-01, Vol.30 (1), p.n/a
Main Authors: Sangat, Prajwol, Indrawan‐Santiago, Maria, Taniar, David
Format: Article
Language:English
Subjects:
Aerospace industry
Apache Spark
Automation
Data analysis
data ingestion
Data management
Data processing
Data retrieval
Data storage
Industrial robots
Ingestion
Manufacturing engineering
MongoDB
Performance evaluation
Railroad cars
sensor data management
Sensors
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Summary Sensors are widely used in the field of manufacturing, railways, aerospace, cars, medicines, robotics, and many other aspects of our everyday life. There is an increasing need to capture, store, and analyse the dynamic semi‐structured data from those sensors. A similar growth of semi‐structured data in the modern web has led to the creation of NoSQL data stores for scalability, availability, and performance, whereas large‐scale data processing frameworks for parallel analysis. NoSQL data store such as MongoDB and data processing framework such as Apache Hadoop has been studied for scientific data analysis. However, there has been no study on MongoDB with Apache Spark, and there is a limited understanding of how sensor data management can benefit from these technologies, specifically for ingesting high‐velocity sensor data and parallel retrieval of high volume data. In this paper, we evaluate the performance of MongoDB sharding and no‐sharding databases with Apache Spark, to identify the right software environment for sensor data management.
ISSN:1532-0626
1532-0634
DOI:10.1002/cpe.4354