Entropy and sigmoid based K-means clustering and AGWO for effective big data handling

In this article, we have presented the effective handling of big data using adaptive clustering and optimization techniques. Initially, heterogeneous data is collected from multiple sources and then transformed the data into desired network graphs. Then finding patterns in the graphs, the module dis...

Full description

Saved in:
Bibliographic Details
Published in:Multimedia tools and applications 2023-04, Vol.82 (10), p.15287-15304
Main Authors: Vankdothu, Ramdas, Hameed, Mohd Abdul, Bhukya, Raju, Garg, Gaurav
Format: Article
Language:English
Subjects:
Algorithms
Big Data
Cluster analysis
Clustering
Entropy
Graphs
Optimization
Optimization techniques
Queries
Vector quantization
Citations: Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this article, we have presented the effective handling of big data using adaptive clustering and optimization techniques. Initially, heterogeneous data is collected from multiple sources and then transformed the data into desired network graphs. Then finding patterns in the graphs, the module distributes the data into the right data blocks using Entropy and sigmoid based K-means clustering. Subsequently, an adaptive grey wolf optimization (AGWO) algorithm in Hadoop distributed file system (HDFS) distributes the data blocks into the right machine. This optimized HDFS serves as a data source for services to execute queries and provide a platform to apply graph algorithms efficiently as well as reduce resource usage. Finally, we can handle a broad range of data types, query time, and resource usage. The experimental results of the proposed work provide better results in comparison with the existing methods such as GWO and PSOin terms of the algorithm run Time, loading time, resource usage, Query time, Query execution time and convergence.
ISSN:1380-7501
1573-7721
DOI:10.1007/s11042-022-13929-2