Optimizing task allocation in multi-query edge analytics

• Published in: Cluster computing 2024-09, Vol.27 (6), p.8289-8306
• Main Authors: Michailidou, Anna-Valentini, Bellas, Christos, Gounaris, Anastasios
• Format: Article
• Language: English
• Subjects: Algorithms; Computer Communication Networks; Computer Science; Data analysis; Dynamic programming; Edge computing; Internet of Things; Operating Systems; Optimization; Placement; Processor Architectures; Prototypes; Queries; Query processing; Resource scheduling; Resource utilization; Sensors; Smartwatches; Wearable computers
• ISSN: 1386-7857; 1573-7543
• DOI: 10.1007/s10586-024-04427-1

Edge analytics receives an ever-increasing interest since processing streaming data closer to where they are produced, rather than transferring them to the cloud, ensures lower latency while also addresses data privacy issues. In this work, we deal with the placement of analytic tasks to heterogeneous geo-distributed edge devices while targeting three objectives, namely latency, quality of results, and resource utilization. In addition, we investigate this multi-objective problem in a multi-query setting, where we jointly optimize multiple analytic jobs while dynamically adjusting task placement decisions. We explore multiple solutions that we thoroughly evaluate; interestingly, in a multi-query setting, all three objectives can be improved simultaneously by our proposals in many cases. Furthermore, we develop a proof-of-concept prototype using Apache Storm. Our solutions are thoroughly evaluated and shown to yield improvements by more than 50% compared to advanced baselines targeting only latency. Moreover, our software prototype managed to achieve speedups of up to 6 × over the Resource Aware Apache Storm scheduler, with an average speedup of 2.76 × , when deployed over a small-scale infrastructure.