Loading…

Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoT

Deep reinforcement learning (DRL) is a promising technology to address the resource allocation problem for efficient data transmission in complex network environments. However, most DRL-based resource allocation algorithms suffer from limited feature extraction capabilities and lack scalability and...

Full description

Saved in:
Bibliographic Details
Published in:Information fusion 2023-11, Vol.99, p.101871, Article 101871
Main Authors: Wang, Ying, Shang, Fengjun, Lei, Jianjun
Format: Article
Language:English
Subjects:
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c306t-91007612e3bd134a8d56a170947684313c5acddc737742272d5bb39f5cb3b3ce3
cites cdi_FETCH-LOGICAL-c306t-91007612e3bd134a8d56a170947684313c5acddc737742272d5bb39f5cb3b3ce3
container_end_page
container_issue
container_start_page 101871
container_title Information fusion
container_volume 99
creator Wang, Ying
Shang, Fengjun
Lei, Jianjun
description Deep reinforcement learning (DRL) is a promising technology to address the resource allocation problem for efficient data transmission in complex network environments. However, most DRL-based resource allocation algorithms suffer from limited feature extraction capabilities and lack scalability and generalization, especially in heterogeneous Industrial Internet of Things (IIoT) environments. In this paper, we develop a lifelong learning architecture that can integrate artificial intelligence (AI) algorithms into the heterogeneous IIoT network for efficient data transmission. Based on this, we propose an intelligent resource allocation algorithm based on dual-attention DRL (DADR) for forwarding node selection and channel access slot allocation in a specific network environment. The proposed DADR algorithm combines the advantages of multi-dimension convolutional attention and multi-head self-attention mechanisms. It can provide local- and global-feature fusion capabilities for distributed nodes while maximizing the performance of data transmission. Furthermore, we present a lifelong federated meta reinforcement learning (LFMRL) that can effectively utilize prior knowledge and enable the DRL agent quickly adapt to a new environment. Specifically, LFMRL adopts a federated meta learning-based knowledge fusion algorithm to fuse the knowledge of learned DADR algorithms and iteratively update the shared model, thereby improving the scalability and generalization of the shared model in heterogeneous IIoT environments. In addition, a simple and efficient knowledge transfer mechanism is enabled to accelerate the DRL model convergence by transferring the knowledge of the shared model to the new environment. Simulation results demonstrate the effectiveness of the proposed algorithms in terms of energy efficiency, data transmission reliability, and network stability. Compared to DADR and FedAvg algorithms, LFMRL algorithm can further reduce the energy consumption, training time, and average forwarding node switching times, while improving packet delivery rate to 99.2%. •A novel lifelong learning architecture for resource allocation in heterogeneous IIoT.•Dual-attention deep reinforcement learning provides local- and global- feature fusion.•Federated meta learning based knowledge fusion learns the commonness of models.•Reinforcement learning based knowledge transfer learns the properties of model.•Knowledge fusion and knowledge transfer provide lifelong learning ability
doi_str_mv 10.1016/j.inffus.2023.101871
format article
fullrecord <record><control><sourceid>elsevier_cross</sourceid><recordid>TN_cdi_crossref_primary_10_1016_j_inffus_2023_101871</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S1566253523001872</els_id><sourcerecordid>S1566253523001872</sourcerecordid><originalsourceid>FETCH-LOGICAL-c306t-91007612e3bd134a8d56a170947684313c5acddc737742272d5bb39f5cb3b3ce3</originalsourceid><addsrcrecordid>eNp9kE1OwzAQhSMEEqVwAxa-QIodx067QUIVP5VAbMracuxJ6sq1K9tB6nm4KA5BYsdqZj7Ne5p5RXFL8IJgwu_2C-O6boiLCld0RMuGnBWzXKqSU8zOc884LytG2WVxFeMeY9JgSmbF19tgkyn7IN1gZTDphLKR8Q4FiH4ICpC01iuZRiZt7_PO7oBaGUGjjPQgbSlTAvezoQGOWZrv8Vl7yBRZkMEZ1yPpNLKmA-vz8EeD2pkEKg0BkHFoBwmC78GBHyLabPz2urjopI1w81vnxcfT43b9Ur6-P2_WD6-lopinckUwbjipgLaa0FouNeMyv7mqG76sKaGKSaW1amjT1FXVVJq1LV11TLW0pQrovKgnXxV8jAE6cQzmIMNJECzGoMVeTEGLMWgxBZ1l95MM8m2fBoKIyoBToE3Ibwntzf8G31bojdM</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoT</title><source>ScienceDirect Freedom Collection 2022-2024</source><creator>Wang, Ying ; Shang, Fengjun ; Lei, Jianjun</creator><creatorcontrib>Wang, Ying ; Shang, Fengjun ; Lei, Jianjun</creatorcontrib><description>Deep reinforcement learning (DRL) is a promising technology to address the resource allocation problem for efficient data transmission in complex network environments. However, most DRL-based resource allocation algorithms suffer from limited feature extraction capabilities and lack scalability and generalization, especially in heterogeneous Industrial Internet of Things (IIoT) environments. In this paper, we develop a lifelong learning architecture that can integrate artificial intelligence (AI) algorithms into the heterogeneous IIoT network for efficient data transmission. Based on this, we propose an intelligent resource allocation algorithm based on dual-attention DRL (DADR) for forwarding node selection and channel access slot allocation in a specific network environment. The proposed DADR algorithm combines the advantages of multi-dimension convolutional attention and multi-head self-attention mechanisms. It can provide local- and global-feature fusion capabilities for distributed nodes while maximizing the performance of data transmission. Furthermore, we present a lifelong federated meta reinforcement learning (LFMRL) that can effectively utilize prior knowledge and enable the DRL agent quickly adapt to a new environment. Specifically, LFMRL adopts a federated meta learning-based knowledge fusion algorithm to fuse the knowledge of learned DADR algorithms and iteratively update the shared model, thereby improving the scalability and generalization of the shared model in heterogeneous IIoT environments. In addition, a simple and efficient knowledge transfer mechanism is enabled to accelerate the DRL model convergence by transferring the knowledge of the shared model to the new environment. Simulation results demonstrate the effectiveness of the proposed algorithms in terms of energy efficiency, data transmission reliability, and network stability. Compared to DADR and FedAvg algorithms, LFMRL algorithm can further reduce the energy consumption, training time, and average forwarding node switching times, while improving packet delivery rate to 99.2%. •A novel lifelong learning architecture for resource allocation in heterogeneous IIoT.•Dual-attention deep reinforcement learning provides local- and global- feature fusion.•Federated meta learning based knowledge fusion learns the commonness of models.•Reinforcement learning based knowledge transfer learns the properties of model.•Knowledge fusion and knowledge transfer provide lifelong learning ability for system.</description><identifier>ISSN: 1566-2535</identifier><identifier>EISSN: 1872-6305</identifier><identifier>DOI: 10.1016/j.inffus.2023.101871</identifier><language>eng</language><publisher>Elsevier B.V</publisher><subject>Deep reinforcement learning ; Federated meta learning ; Heterogeneous industrial Internet of Things ; Lifelong learning ; Resource allocation</subject><ispartof>Information fusion, 2023-11, Vol.99, p.101871, Article 101871</ispartof><rights>2023 Elsevier B.V.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c306t-91007612e3bd134a8d56a170947684313c5acddc737742272d5bb39f5cb3b3ce3</citedby><cites>FETCH-LOGICAL-c306t-91007612e3bd134a8d56a170947684313c5acddc737742272d5bb39f5cb3b3ce3</cites><orcidid>0000-0001-8454-3413</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,776,780,27901,27902</link.rule.ids></links><search><creatorcontrib>Wang, Ying</creatorcontrib><creatorcontrib>Shang, Fengjun</creatorcontrib><creatorcontrib>Lei, Jianjun</creatorcontrib><title>Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoT</title><title>Information fusion</title><description>Deep reinforcement learning (DRL) is a promising technology to address the resource allocation problem for efficient data transmission in complex network environments. However, most DRL-based resource allocation algorithms suffer from limited feature extraction capabilities and lack scalability and generalization, especially in heterogeneous Industrial Internet of Things (IIoT) environments. In this paper, we develop a lifelong learning architecture that can integrate artificial intelligence (AI) algorithms into the heterogeneous IIoT network for efficient data transmission. Based on this, we propose an intelligent resource allocation algorithm based on dual-attention DRL (DADR) for forwarding node selection and channel access slot allocation in a specific network environment. The proposed DADR algorithm combines the advantages of multi-dimension convolutional attention and multi-head self-attention mechanisms. It can provide local- and global-feature fusion capabilities for distributed nodes while maximizing the performance of data transmission. Furthermore, we present a lifelong federated meta reinforcement learning (LFMRL) that can effectively utilize prior knowledge and enable the DRL agent quickly adapt to a new environment. Specifically, LFMRL adopts a federated meta learning-based knowledge fusion algorithm to fuse the knowledge of learned DADR algorithms and iteratively update the shared model, thereby improving the scalability and generalization of the shared model in heterogeneous IIoT environments. In addition, a simple and efficient knowledge transfer mechanism is enabled to accelerate the DRL model convergence by transferring the knowledge of the shared model to the new environment. Simulation results demonstrate the effectiveness of the proposed algorithms in terms of energy efficiency, data transmission reliability, and network stability. Compared to DADR and FedAvg algorithms, LFMRL algorithm can further reduce the energy consumption, training time, and average forwarding node switching times, while improving packet delivery rate to 99.2%. •A novel lifelong learning architecture for resource allocation in heterogeneous IIoT.•Dual-attention deep reinforcement learning provides local- and global- feature fusion.•Federated meta learning based knowledge fusion learns the commonness of models.•Reinforcement learning based knowledge transfer learns the properties of model.•Knowledge fusion and knowledge transfer provide lifelong learning ability for system.</description><subject>Deep reinforcement learning</subject><subject>Federated meta learning</subject><subject>Heterogeneous industrial Internet of Things</subject><subject>Lifelong learning</subject><subject>Resource allocation</subject><issn>1566-2535</issn><issn>1872-6305</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><recordid>eNp9kE1OwzAQhSMEEqVwAxa-QIodx067QUIVP5VAbMracuxJ6sq1K9tB6nm4KA5BYsdqZj7Ne5p5RXFL8IJgwu_2C-O6boiLCld0RMuGnBWzXKqSU8zOc884LytG2WVxFeMeY9JgSmbF19tgkyn7IN1gZTDphLKR8Q4FiH4ICpC01iuZRiZt7_PO7oBaGUGjjPQgbSlTAvezoQGOWZrv8Vl7yBRZkMEZ1yPpNLKmA-vz8EeD2pkEKg0BkHFoBwmC78GBHyLabPz2urjopI1w81vnxcfT43b9Ur6-P2_WD6-lopinckUwbjipgLaa0FouNeMyv7mqG76sKaGKSaW1amjT1FXVVJq1LV11TLW0pQrovKgnXxV8jAE6cQzmIMNJECzGoMVeTEGLMWgxBZ1l95MM8m2fBoKIyoBToE3Ibwntzf8G31bojdM</recordid><startdate>202311</startdate><enddate>202311</enddate><creator>Wang, Ying</creator><creator>Shang, Fengjun</creator><creator>Lei, Jianjun</creator><general>Elsevier B.V</general><scope>AAYXX</scope><scope>CITATION</scope><orcidid>https://orcid.org/0000-0001-8454-3413</orcidid></search><sort><creationdate>202311</creationdate><title>Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoT</title><author>Wang, Ying ; Shang, Fengjun ; Lei, Jianjun</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c306t-91007612e3bd134a8d56a170947684313c5acddc737742272d5bb39f5cb3b3ce3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Deep reinforcement learning</topic><topic>Federated meta learning</topic><topic>Heterogeneous industrial Internet of Things</topic><topic>Lifelong learning</topic><topic>Resource allocation</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Wang, Ying</creatorcontrib><creatorcontrib>Shang, Fengjun</creatorcontrib><creatorcontrib>Lei, Jianjun</creatorcontrib><collection>CrossRef</collection><jtitle>Information fusion</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Wang, Ying</au><au>Shang, Fengjun</au><au>Lei, Jianjun</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoT</atitle><jtitle>Information fusion</jtitle><date>2023-11</date><risdate>2023</risdate><volume>99</volume><spage>101871</spage><pages>101871-</pages><artnum>101871</artnum><issn>1566-2535</issn><eissn>1872-6305</eissn><abstract>Deep reinforcement learning (DRL) is a promising technology to address the resource allocation problem for efficient data transmission in complex network environments. However, most DRL-based resource allocation algorithms suffer from limited feature extraction capabilities and lack scalability and generalization, especially in heterogeneous Industrial Internet of Things (IIoT) environments. In this paper, we develop a lifelong learning architecture that can integrate artificial intelligence (AI) algorithms into the heterogeneous IIoT network for efficient data transmission. Based on this, we propose an intelligent resource allocation algorithm based on dual-attention DRL (DADR) for forwarding node selection and channel access slot allocation in a specific network environment. The proposed DADR algorithm combines the advantages of multi-dimension convolutional attention and multi-head self-attention mechanisms. It can provide local- and global-feature fusion capabilities for distributed nodes while maximizing the performance of data transmission. Furthermore, we present a lifelong federated meta reinforcement learning (LFMRL) that can effectively utilize prior knowledge and enable the DRL agent quickly adapt to a new environment. Specifically, LFMRL adopts a federated meta learning-based knowledge fusion algorithm to fuse the knowledge of learned DADR algorithms and iteratively update the shared model, thereby improving the scalability and generalization of the shared model in heterogeneous IIoT environments. In addition, a simple and efficient knowledge transfer mechanism is enabled to accelerate the DRL model convergence by transferring the knowledge of the shared model to the new environment. Simulation results demonstrate the effectiveness of the proposed algorithms in terms of energy efficiency, data transmission reliability, and network stability. Compared to DADR and FedAvg algorithms, LFMRL algorithm can further reduce the energy consumption, training time, and average forwarding node switching times, while improving packet delivery rate to 99.2%. •A novel lifelong learning architecture for resource allocation in heterogeneous IIoT.•Dual-attention deep reinforcement learning provides local- and global- feature fusion.•Federated meta learning based knowledge fusion learns the commonness of models.•Reinforcement learning based knowledge transfer learns the properties of model.•Knowledge fusion and knowledge transfer provide lifelong learning ability for system.</abstract><pub>Elsevier B.V</pub><doi>10.1016/j.inffus.2023.101871</doi><orcidid>https://orcid.org/0000-0001-8454-3413</orcidid></addata></record>
fulltext fulltext
identifier ISSN: 1566-2535
ispartof Information fusion, 2023-11, Vol.99, p.101871, Article 101871
issn 1566-2535
1872-6305
language eng
recordid cdi_crossref_primary_10_1016_j_inffus_2023_101871
source ScienceDirect Freedom Collection 2022-2024
subjects Deep reinforcement learning
Federated meta learning
Heterogeneous industrial Internet of Things
Lifelong learning
Resource allocation
title Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoT
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-16T03%3A33%3A24IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-elsevier_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Multi-granularity%20fusion%20resource%20allocation%20algorithm%20based%20on%20dual-attention%20deep%20reinforcement%20learning%20and%20lifelong%20learning%20architecture%20in%20heterogeneous%20IIoT&rft.jtitle=Information%20fusion&rft.au=Wang,%20Ying&rft.date=2023-11&rft.volume=99&rft.spage=101871&rft.pages=101871-&rft.artnum=101871&rft.issn=1566-2535&rft.eissn=1872-6305&rft_id=info:doi/10.1016/j.inffus.2023.101871&rft_dat=%3Celsevier_cross%3ES1566253523001872%3C/elsevier_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c306t-91007612e3bd134a8d56a170947684313c5acddc737742272d5bb39f5cb3b3ce3%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true