Design and implementation of a cloud-based event-driven architecture for real-time data processing in wireless sensor networks

The growth of the Internet of Things (IoTs) and the number of connected devices is driven by emerging applications and business models. One common aim is to provide systems able to synchronize these devices, handle the big amount of daily generated data and meet business demands. This paper proposes...

Full description

Saved in:
Bibliographic Details
Published in:The Journal of supercomputing 2022-02, Vol.78 (3), p.3374-3401
Main Authors: Khriji, Sabrine, Benbelgacem, Yahia, Chéour, Rym, Houssaini, Dhouha El, Kanoun, Olfa
Format: Article
Language:English
Subjects:
Cloud computing
Compilers
Computer architecture
Computer Science
Data processing
Internet of Things
Interpreters
Load tests
Network latency
Processor Architectures
Programming Languages
Real time
Web services
Wireless networks
Wireless sensor networks
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The growth of the Internet of Things (IoTs) and the number of connected devices is driven by emerging applications and business models. One common aim is to provide systems able to synchronize these devices, handle the big amount of daily generated data and meet business demands. This paper proposes a cost-effective cloud-based architecture using an event-driven backbone to process many applications’ data in real-time, called REDA. It supports the Amazon Web Service (AWS) IoT core, and it opens the door as a free software-based implementation. Measured data from several wireless sensor nodes are transmitted to the cloud running application through the lightweight publisher/subscriber messaging transport protocol, MQTT. The real-time stream processing platform, Apache Kafka, is used as a message broker to receive data from the producer and forward it to the correspondent consumer. Micro-services design patterns, as an event consumer, are implemented with Java spring and managed with Apache Maven to avoid the monolithic applications’ problem. The Apache Kafka cluster co-located with Zookeeper is deployed over three availability zones and optimized for high throughput and low latency. To guarantee no message loss and to simulate the system performances, different load tests are carried out. The proposed architecture is reliable in stress cases and can handle records goes to 8000 messages in a second with low latency in a cheap hosted and configured architecture.
ISSN:0920-8542
1573-0484
DOI:10.1007/s11227-021-03955-6