Loading…
Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoT
Deep reinforcement learning (DRL) is a promising technology to address the resource allocation problem for efficient data transmission in complex network environments. However, most DRL-based resource allocation algorithms suffer from limited feature extraction capabilities and lack scalability and...
Saved in:
Published in: | Information fusion 2023-11, Vol.99, p.101871, Article 101871 |
---|---|
Main Authors: | , , |
Format: | Article |
Language: | English |
Subjects: | |
Citations: | Items that this one cites Items that cite this one |
Online Access: | Get full text |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
cited_by | cdi_FETCH-LOGICAL-c306t-91007612e3bd134a8d56a170947684313c5acddc737742272d5bb39f5cb3b3ce3 |
---|---|
cites | cdi_FETCH-LOGICAL-c306t-91007612e3bd134a8d56a170947684313c5acddc737742272d5bb39f5cb3b3ce3 |
container_end_page | |
container_issue | |
container_start_page | 101871 |
container_title | Information fusion |
container_volume | 99 |
creator | Wang, Ying Shang, Fengjun Lei, Jianjun |
description | Deep reinforcement learning (DRL) is a promising technology to address the resource allocation problem for efficient data transmission in complex network environments. However, most DRL-based resource allocation algorithms suffer from limited feature extraction capabilities and lack scalability and generalization, especially in heterogeneous Industrial Internet of Things (IIoT) environments. In this paper, we develop a lifelong learning architecture that can integrate artificial intelligence (AI) algorithms into the heterogeneous IIoT network for efficient data transmission. Based on this, we propose an intelligent resource allocation algorithm based on dual-attention DRL (DADR) for forwarding node selection and channel access slot allocation in a specific network environment. The proposed DADR algorithm combines the advantages of multi-dimension convolutional attention and multi-head self-attention mechanisms. It can provide local- and global-feature fusion capabilities for distributed nodes while maximizing the performance of data transmission. Furthermore, we present a lifelong federated meta reinforcement learning (LFMRL) that can effectively utilize prior knowledge and enable the DRL agent quickly adapt to a new environment. Specifically, LFMRL adopts a federated meta learning-based knowledge fusion algorithm to fuse the knowledge of learned DADR algorithms and iteratively update the shared model, thereby improving the scalability and generalization of the shared model in heterogeneous IIoT environments. In addition, a simple and efficient knowledge transfer mechanism is enabled to accelerate the DRL model convergence by transferring the knowledge of the shared model to the new environment. Simulation results demonstrate the effectiveness of the proposed algorithms in terms of energy efficiency, data transmission reliability, and network stability. Compared to DADR and FedAvg algorithms, LFMRL algorithm can further reduce the energy consumption, training time, and average forwarding node switching times, while improving packet delivery rate to 99.2%.
•A novel lifelong learning architecture for resource allocation in heterogeneous IIoT.•Dual-attention deep reinforcement learning provides local- and global- feature fusion.•Federated meta learning based knowledge fusion learns the commonness of models.•Reinforcement learning based knowledge transfer learns the properties of model.•Knowledge fusion and knowledge transfer provide lifelong learning ability |
doi_str_mv | 10.1016/j.inffus.2023.101871 |
format | article |
fullrecord | <record><control><sourceid>elsevier_cross</sourceid><recordid>TN_cdi_crossref_primary_10_1016_j_inffus_2023_101871</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S1566253523001872</els_id><sourcerecordid>S1566253523001872</sourcerecordid><originalsourceid>FETCH-LOGICAL-c306t-91007612e3bd134a8d56a170947684313c5acddc737742272d5bb39f5cb3b3ce3</originalsourceid><addsrcrecordid>eNp9kE1OwzAQhSMEEqVwAxa-QIodx067QUIVP5VAbMracuxJ6sq1K9tB6nm4KA5BYsdqZj7Ne5p5RXFL8IJgwu_2C-O6boiLCld0RMuGnBWzXKqSU8zOc884LytG2WVxFeMeY9JgSmbF19tgkyn7IN1gZTDphLKR8Q4FiH4ICpC01iuZRiZt7_PO7oBaGUGjjPQgbSlTAvezoQGOWZrv8Vl7yBRZkMEZ1yPpNLKmA-vz8EeD2pkEKg0BkHFoBwmC78GBHyLabPz2urjopI1w81vnxcfT43b9Ur6-P2_WD6-lopinckUwbjipgLaa0FouNeMyv7mqG76sKaGKSaW1amjT1FXVVJq1LV11TLW0pQrovKgnXxV8jAE6cQzmIMNJECzGoMVeTEGLMWgxBZ1l95MM8m2fBoKIyoBToE3Ibwntzf8G31bojdM</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoT</title><source>ScienceDirect Freedom Collection</source><creator>Wang, Ying ; Shang, Fengjun ; Lei, Jianjun</creator><creatorcontrib>Wang, Ying ; Shang, Fengjun ; Lei, Jianjun</creatorcontrib><description>Deep reinforcement learning (DRL) is a promising technology to address the resource allocation problem for efficient data transmission in complex network environments. However, most DRL-based resource allocation algorithms suffer from limited feature extraction capabilities and lack scalability and generalization, especially in heterogeneous Industrial Internet of Things (IIoT) environments. In this paper, we develop a lifelong learning architecture that can integrate artificial intelligence (AI) algorithms into the heterogeneous IIoT network for efficient data transmission. Based on this, we propose an intelligent resource allocation algorithm based on dual-attention DRL (DADR) for forwarding node selection and channel access slot allocation in a specific network environment. The proposed DADR algorithm combines the advantages of multi-dimension convolutional attention and multi-head self-attention mechanisms. It can provide local- and global-feature fusion capabilities for distributed nodes while maximizing the performance of data transmission. Furthermore, we present a lifelong federated meta reinforcement learning (LFMRL) that can effectively utilize prior knowledge and enable the DRL agent quickly adapt to a new environment. Specifically, LFMRL adopts a federated meta learning-based knowledge fusion algorithm to fuse the knowledge of learned DADR algorithms and iteratively update the shared model, thereby improving the scalability and generalization of the shared model in heterogeneous IIoT environments. In addition, a simple and efficient knowledge transfer mechanism is enabled to accelerate the DRL model convergence by transferring the knowledge of the shared model to the new environment. Simulation results demonstrate the effectiveness of the proposed algorithms in terms of energy efficiency, data transmission reliability, and network stability. Compared to DADR and FedAvg algorithms, LFMRL algorithm can further reduce the energy consumption, training time, and average forwarding node switching times, while improving packet delivery rate to 99.2%.
•A novel lifelong learning architecture for resource allocation in heterogeneous IIoT.•Dual-attention deep reinforcement learning provides local- and global- feature fusion.•Federated meta learning based knowledge fusion learns the commonness of models.•Reinforcement learning based knowledge transfer learns the properties of model.•Knowledge fusion and knowledge transfer provide lifelong learning ability for system.</description><identifier>ISSN: 1566-2535</identifier><identifier>EISSN: 1872-6305</identifier><identifier>DOI: 10.1016/j.inffus.2023.101871</identifier><language>eng</language><publisher>Elsevier B.V</publisher><subject>Deep reinforcement learning ; Federated meta learning ; Heterogeneous industrial Internet of Things ; Lifelong learning ; Resource allocation</subject><ispartof>Information fusion, 2023-11, Vol.99, p.101871, Article 101871</ispartof><rights>2023 Elsevier B.V.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c306t-91007612e3bd134a8d56a170947684313c5acddc737742272d5bb39f5cb3b3ce3</citedby><cites>FETCH-LOGICAL-c306t-91007612e3bd134a8d56a170947684313c5acddc737742272d5bb39f5cb3b3ce3</cites><orcidid>0000-0001-8454-3413</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,776,780,27901,27902</link.rule.ids></links><search><creatorcontrib>Wang, Ying</creatorcontrib><creatorcontrib>Shang, Fengjun</creatorcontrib><creatorcontrib>Lei, Jianjun</creatorcontrib><title>Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoT</title><title>Information fusion</title><description>Deep reinforcement learning (DRL) is a promising technology to address the resource allocation problem for efficient data transmission in complex network environments. However, most DRL-based resource allocation algorithms suffer from limited feature extraction capabilities and lack scalability and generalization, especially in heterogeneous Industrial Internet of Things (IIoT) environments. In this paper, we develop a lifelong learning architecture that can integrate artificial intelligence (AI) algorithms into the heterogeneous IIoT network for efficient data transmission. Based on this, we propose an intelligent resource allocation algorithm based on dual-attention DRL (DADR) for forwarding node selection and channel access slot allocation in a specific network environment. The proposed DADR algorithm combines the advantages of multi-dimension convolutional attention and multi-head self-attention mechanisms. It can provide local- and global-feature fusion capabilities for distributed nodes while maximizing the performance of data transmission. Furthermore, we present a lifelong federated meta reinforcement learning (LFMRL) that can effectively utilize prior knowledge and enable the DRL agent quickly adapt to a new environment. Specifically, LFMRL adopts a federated meta learning-based knowledge fusion algorithm to fuse the knowledge of learned DADR algorithms and iteratively update the shared model, thereby improving the scalability and generalization of the shared model in heterogeneous IIoT environments. In addition, a simple and efficient knowledge transfer mechanism is enabled to accelerate the DRL model convergence by transferring the knowledge of the shared model to the new environment. Simulation results demonstrate the effectiveness of the proposed algorithms in terms of energy efficiency, data transmission reliability, and network stability. Compared to DADR and FedAvg algorithms, LFMRL algorithm can further reduce the energy consumption, training time, and average forwarding node switching times, while improving packet delivery rate to 99.2%.
•A novel lifelong learning architecture for resource allocation in heterogeneous IIoT.•Dual-attention deep reinforcement learning provides local- and global- feature fusion.•Federated meta learning based knowledge fusion learns the commonness of models.•Reinforcement learning based knowledge transfer learns the properties of model.•Knowledge fusion and knowledge transfer provide lifelong learning ability for system.</description><subject>Deep reinforcement learning</subject><subject>Federated meta learning</subject><subject>Heterogeneous industrial Internet of Things</subject><subject>Lifelong learning</subject><subject>Resource allocation</subject><issn>1566-2535</issn><issn>1872-6305</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><recordid>eNp9kE1OwzAQhSMEEqVwAxa-QIodx067QUIVP5VAbMracuxJ6sq1K9tB6nm4KA5BYsdqZj7Ne5p5RXFL8IJgwu_2C-O6boiLCld0RMuGnBWzXKqSU8zOc884LytG2WVxFeMeY9JgSmbF19tgkyn7IN1gZTDphLKR8Q4FiH4ICpC01iuZRiZt7_PO7oBaGUGjjPQgbSlTAvezoQGOWZrv8Vl7yBRZkMEZ1yPpNLKmA-vz8EeD2pkEKg0BkHFoBwmC78GBHyLabPz2urjopI1w81vnxcfT43b9Ur6-P2_WD6-lopinckUwbjipgLaa0FouNeMyv7mqG76sKaGKSaW1amjT1FXVVJq1LV11TLW0pQrovKgnXxV8jAE6cQzmIMNJECzGoMVeTEGLMWgxBZ1l95MM8m2fBoKIyoBToE3Ibwntzf8G31bojdM</recordid><startdate>202311</startdate><enddate>202311</enddate><creator>Wang, Ying</creator><creator>Shang, Fengjun</creator><creator>Lei, Jianjun</creator><general>Elsevier B.V</general><scope>AAYXX</scope><scope>CITATION</scope><orcidid>https://orcid.org/0000-0001-8454-3413</orcidid></search><sort><creationdate>202311</creationdate><title>Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoT</title><author>Wang, Ying ; Shang, Fengjun ; Lei, Jianjun</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c306t-91007612e3bd134a8d56a170947684313c5acddc737742272d5bb39f5cb3b3ce3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Deep reinforcement learning</topic><topic>Federated meta learning</topic><topic>Heterogeneous industrial Internet of Things</topic><topic>Lifelong learning</topic><topic>Resource allocation</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Wang, Ying</creatorcontrib><creatorcontrib>Shang, Fengjun</creatorcontrib><creatorcontrib>Lei, Jianjun</creatorcontrib><collection>CrossRef</collection><jtitle>Information fusion</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Wang, Ying</au><au>Shang, Fengjun</au><au>Lei, Jianjun</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoT</atitle><jtitle>Information fusion</jtitle><date>2023-11</date><risdate>2023</risdate><volume>99</volume><spage>101871</spage><pages>101871-</pages><artnum>101871</artnum><issn>1566-2535</issn><eissn>1872-6305</eissn><abstract>Deep reinforcement learning (DRL) is a promising technology to address the resource allocation problem for efficient data transmission in complex network environments. However, most DRL-based resource allocation algorithms suffer from limited feature extraction capabilities and lack scalability and generalization, especially in heterogeneous Industrial Internet of Things (IIoT) environments. In this paper, we develop a lifelong learning architecture that can integrate artificial intelligence (AI) algorithms into the heterogeneous IIoT network for efficient data transmission. Based on this, we propose an intelligent resource allocation algorithm based on dual-attention DRL (DADR) for forwarding node selection and channel access slot allocation in a specific network environment. The proposed DADR algorithm combines the advantages of multi-dimension convolutional attention and multi-head self-attention mechanisms. It can provide local- and global-feature fusion capabilities for distributed nodes while maximizing the performance of data transmission. Furthermore, we present a lifelong federated meta reinforcement learning (LFMRL) that can effectively utilize prior knowledge and enable the DRL agent quickly adapt to a new environment. Specifically, LFMRL adopts a federated meta learning-based knowledge fusion algorithm to fuse the knowledge of learned DADR algorithms and iteratively update the shared model, thereby improving the scalability and generalization of the shared model in heterogeneous IIoT environments. In addition, a simple and efficient knowledge transfer mechanism is enabled to accelerate the DRL model convergence by transferring the knowledge of the shared model to the new environment. Simulation results demonstrate the effectiveness of the proposed algorithms in terms of energy efficiency, data transmission reliability, and network stability. Compared to DADR and FedAvg algorithms, LFMRL algorithm can further reduce the energy consumption, training time, and average forwarding node switching times, while improving packet delivery rate to 99.2%.
•A novel lifelong learning architecture for resource allocation in heterogeneous IIoT.•Dual-attention deep reinforcement learning provides local- and global- feature fusion.•Federated meta learning based knowledge fusion learns the commonness of models.•Reinforcement learning based knowledge transfer learns the properties of model.•Knowledge fusion and knowledge transfer provide lifelong learning ability for system.</abstract><pub>Elsevier B.V</pub><doi>10.1016/j.inffus.2023.101871</doi><orcidid>https://orcid.org/0000-0001-8454-3413</orcidid></addata></record> |
fulltext | fulltext |
identifier | ISSN: 1566-2535 |
ispartof | Information fusion, 2023-11, Vol.99, p.101871, Article 101871 |
issn | 1566-2535 1872-6305 |
language | eng |
recordid | cdi_crossref_primary_10_1016_j_inffus_2023_101871 |
source | ScienceDirect Freedom Collection |
subjects | Deep reinforcement learning Federated meta learning Heterogeneous industrial Internet of Things Lifelong learning Resource allocation |
title | Multi-granularity fusion resource allocation algorithm based on dual-attention deep reinforcement learning and lifelong learning architecture in heterogeneous IIoT |
url | http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-16T01%3A47%3A50IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-elsevier_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Multi-granularity%20fusion%20resource%20allocation%20algorithm%20based%20on%20dual-attention%20deep%20reinforcement%20learning%20and%20lifelong%20learning%20architecture%20in%20heterogeneous%20IIoT&rft.jtitle=Information%20fusion&rft.au=Wang,%20Ying&rft.date=2023-11&rft.volume=99&rft.spage=101871&rft.pages=101871-&rft.artnum=101871&rft.issn=1566-2535&rft.eissn=1872-6305&rft_id=info:doi/10.1016/j.inffus.2023.101871&rft_dat=%3Celsevier_cross%3ES1566253523001872%3C/elsevier_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c306t-91007612e3bd134a8d56a170947684313c5acddc737742272d5bb39f5cb3b3ce3%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true |