Mass data processing and multidimensional database management based on deep learning

• Published in: Open computer science 2022-09, Vol.12 (1), p.300-313
• Main Authors: Shen, Haijie, Li, Yangyuan, Tian, Xinzhi, Chen, Xiaofan, Li, Caihong, Bian, Qian, Wang, Zhenduo, Wang, Weihua
• Format: Article
• Language: English
• Subjects: data processing; database management; deep learning; MOLAP model
• ISSN: 2299-1093; 2299-1093
• DOI: 10.1515/comp-2022-0251

With the rapid development of the Internet of Things, the requirements for massive data processing technology are getting higher and higher. Traditional computer data processing capabilities can no longer deliver fast, simple, and efficient data analysis and processing for today’s massive data processing due to the real-time, massive, polymorphic, and heterogeneous characteristics of Internet of Things data. Mass heterogeneous data of different types of subsystems in the Internet of Things need to be processed and stored uniformly, so the mass data processing method is required to be able to integrate multiple different networks, multiple data sources, and heterogeneous mass data and be able to perform processing on these data. Therefore, this article proposes massive data processing and multidimensional database management based on deep learning to meet the needs of contemporary society for massive data processing. This article has deeply studied the basic technical methods of massive data processing, including MapReduce technology, parallel data technology, database technology based on distributed memory databases, and distributed real-time database technology based on cloud computing technology, and constructed a massive data fusion algorithm based on deep learning. The model and the multidimensional online analytical processing model of the multidimensional database based on deep learning analyze the performance, scalability, load balancing, data query, and other aspects of the multidimensional database based on deep learning. It is concluded that the accuracy of multidimensional database query data is as high as 100%, and the accuracy of the average data query time is only 0.0053 s, which is much lower than the general database query time.