Loading…

Complete Stability of Delayed Recurrent Neural Networks With New Wave-Type Activation Functions

Activation functions have a significant effect on the dynamics of neural networks (NNs). This study proposes new nonmonotonic wave-type activation functions and examines the complete stability of delayed recurrent NNs (DRNNs) with these activation functions. Using the geometrical properties of the w...

Full description

Saved in:

Bibliographic Details
Published in:	IEEE transaction on neural networks and learning systems 2024-05, Vol.PP, p.1-13
Main Authors:	Yan, Zepeng, Sun, Wen, Guo, Wanli, Li, Biwen, Wen, Shiping, Cao, Jinde
Format:	Article
Language:	English
Subjects:	Artificial neural networks Complete stability delayed recurrent neural network (DRNN) Delays Neurons Numerical stability Stability criteria Sun Thermal stability time-varying delay wave-type activation function
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

cited_by
cites
container_end_page	13
container_issue
container_start_page	1
container_title	IEEE transaction on neural networks and learning systems
container_volume	PP
creator	Yan, Zepeng Sun, Wen Guo, Wanli Li, Biwen Wen, Shiping Cao, Jinde
description	Activation functions have a significant effect on the dynamics of neural networks (NNs). This study proposes new nonmonotonic wave-type activation functions and examines the complete stability of delayed recurrent NNs (DRNNs) with these activation functions. Using the geometrical properties of the wave-type activation function and subsequent iteration scheme, sufficient conditions are provided to ensure that a DRNN with n neurons has exactly (2m + 3)^n equilibria, where (m + 2)^n equilibria are locally exponentially stable, the remainder (2m + 3)^n - (m + 2)^n equilibria are unstable, and a positive integer m is related to wave-type activation functions. Furthermore, the DRNN with the proposed activation function is completely stable. Compared with the previous literature, the total number of equilibria and the stable equilibria significantly increase, thereby enhancing the memory storage capacity of DRNN. Finally, several examples are presented to demonstrate our proposed results.
doi_str_mv	10.1109/TNNLS.2024.3394854
format	article
fullrecord	<record><control><sourceid>proquest_ieee_</sourceid><recordid>TN_cdi_ieee_primary_10520819</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>10520819</ieee_id><sourcerecordid>3051937817</sourcerecordid><originalsourceid>FETCH-LOGICAL-c275t-941cf0964c41eb23b38e5f0dccddf142eae972c0c7a1f58cc3258f4d31d406cd3</originalsourceid><addsrcrecordid>eNpNkE9Lw0AQxRdRbKn9AiKyRy-p-y_J5liqVaFUsJV6WzabCUbTJu5uWvLtTW0V5_Jm4M2b4YfQJSUjSklyu5zPZ4sRI0yMOE-EDMUJ6jMasYBxKU__-vith4bOfZCuIhJGIjlHPS5jkkQk7iM1qdZ1CR7wwuu0KAvf4irHd1DqFjL8AqaxFjYez6GxuuzE7yr76fCq8O_dtMMrvYVg2daAx8YXW-2LaoOnzcbsG3eBznJdOhgedYBep_fLyWMwe354moxngWFx6INEUJN3LwkjKKSMp1xCmJPMmCzLqWCgIYmZISbWNA-lMZyFMhcZp5kgkcn4AN0ccmtbfTXgvFoXzkBZ6g1UjVOchDThsaRxZ2UHq7GVcxZyVdtirW2rKFF7tuqHrdqzVUe23dL1Mb9J15D9rfyS7AxXB0MBAP8SQ0Zkd_kbmLt-7w</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3051937817</pqid></control><display><type>article</type><title>Complete Stability of Delayed Recurrent Neural Networks With New Wave-Type Activation Functions</title><source>IEEE Electronic Library (IEL) Journals</source><creator>Yan, Zepeng ; Sun, Wen ; Guo, Wanli ; Li, Biwen ; Wen, Shiping ; Cao, Jinde</creator><creatorcontrib>Yan, Zepeng ; Sun, Wen ; Guo, Wanli ; Li, Biwen ; Wen, Shiping ; Cao, Jinde</creatorcontrib><description><![CDATA[Activation functions have a significant effect on the dynamics of neural networks (NNs). This study proposes new nonmonotonic wave-type activation functions and examines the complete stability of delayed recurrent NNs (DRNNs) with these activation functions. Using the geometrical properties of the wave-type activation function and subsequent iteration scheme, sufficient conditions are provided to ensure that a DRNN with n neurons has exactly <inline-formula> <tex-math notation="LaTeX">(2m</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">+</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">3)^n</tex-math> </inline-formula> equilibria, where <inline-formula> <tex-math notation="LaTeX">(m</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">+</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">2)^n</tex-math> </inline-formula> equilibria are locally exponentially stable, the remainder <inline-formula> <tex-math notation="LaTeX">(2m</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">+</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">3)^n</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">-</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">(m</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">+</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">2)^n</tex-math> </inline-formula> equilibria are unstable, and a positive integer <inline-formula> <tex-math notation="LaTeX">m</tex-math> </inline-formula> is related to wave-type activation functions. Furthermore, the DRNN with the proposed activation function is completely stable. Compared with the previous literature, the total number of equilibria and the stable equilibria significantly increase, thereby enhancing the memory storage capacity of DRNN. Finally, several examples are presented to demonstrate our proposed results.]]></description><identifier>ISSN: 2162-237X</identifier><identifier>EISSN: 2162-2388</identifier><identifier>DOI: 10.1109/TNNLS.2024.3394854</identifier><identifier>PMID: 38709607</identifier><identifier>CODEN: ITNNAL</identifier><language>eng</language><publisher>United States: IEEE</publisher><subject>Artificial neural networks ; Complete stability ; delayed recurrent neural network (DRNN) ; Delays ; Neurons ; Numerical stability ; Stability criteria ; Sun ; Thermal stability ; time-varying delay ; wave-type activation function</subject><ispartof>IEEE transaction on neural networks and learning systems, 2024-05, Vol.PP, p.1-13</ispartof><woscitedreferencessubscribed>false</woscitedreferencessubscribed><orcidid>0009-0007-5291-9472 ; 0000-0002-5048-0319 ; 0000-0001-7070-4730 ; 0000-0003-0312-7464 ; 0000-0003-3133-7119 ; 0000-0003-2928-6703</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/10520819$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,776,780,27900,27901,54770</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/38709607$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Yan, Zepeng</creatorcontrib><creatorcontrib>Sun, Wen</creatorcontrib><creatorcontrib>Guo, Wanli</creatorcontrib><creatorcontrib>Li, Biwen</creatorcontrib><creatorcontrib>Wen, Shiping</creatorcontrib><creatorcontrib>Cao, Jinde</creatorcontrib><title>Complete Stability of Delayed Recurrent Neural Networks With New Wave-Type Activation Functions</title><title>IEEE transaction on neural networks and learning systems</title><addtitle>TNNLS</addtitle><addtitle>IEEE Trans Neural Netw Learn Syst</addtitle><description><![CDATA[Activation functions have a significant effect on the dynamics of neural networks (NNs). This study proposes new nonmonotonic wave-type activation functions and examines the complete stability of delayed recurrent NNs (DRNNs) with these activation functions. Using the geometrical properties of the wave-type activation function and subsequent iteration scheme, sufficient conditions are provided to ensure that a DRNN with n neurons has exactly <inline-formula> <tex-math notation="LaTeX">(2m</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">+</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">3)^n</tex-math> </inline-formula> equilibria, where <inline-formula> <tex-math notation="LaTeX">(m</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">+</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">2)^n</tex-math> </inline-formula> equilibria are locally exponentially stable, the remainder <inline-formula> <tex-math notation="LaTeX">(2m</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">+</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">3)^n</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">-</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">(m</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">+</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">2)^n</tex-math> </inline-formula> equilibria are unstable, and a positive integer <inline-formula> <tex-math notation="LaTeX">m</tex-math> </inline-formula> is related to wave-type activation functions. Furthermore, the DRNN with the proposed activation function is completely stable. Compared with the previous literature, the total number of equilibria and the stable equilibria significantly increase, thereby enhancing the memory storage capacity of DRNN. Finally, several examples are presented to demonstrate our proposed results.]]></description><subject>Artificial neural networks</subject><subject>Complete stability</subject><subject>delayed recurrent neural network (DRNN)</subject><subject>Delays</subject><subject>Neurons</subject><subject>Numerical stability</subject><subject>Stability criteria</subject><subject>Sun</subject><subject>Thermal stability</subject><subject>time-varying delay</subject><subject>wave-type activation function</subject><issn>2162-237X</issn><issn>2162-2388</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><recordid>eNpNkE9Lw0AQxRdRbKn9AiKyRy-p-y_J5liqVaFUsJV6WzabCUbTJu5uWvLtTW0V5_Jm4M2b4YfQJSUjSklyu5zPZ4sRI0yMOE-EDMUJ6jMasYBxKU__-vith4bOfZCuIhJGIjlHPS5jkkQk7iM1qdZ1CR7wwuu0KAvf4irHd1DqFjL8AqaxFjYez6GxuuzE7yr76fCq8O_dtMMrvYVg2daAx8YXW-2LaoOnzcbsG3eBznJdOhgedYBep_fLyWMwe354moxngWFx6INEUJN3LwkjKKSMp1xCmJPMmCzLqWCgIYmZISbWNA-lMZyFMhcZp5kgkcn4AN0ccmtbfTXgvFoXzkBZ6g1UjVOchDThsaRxZ2UHq7GVcxZyVdtirW2rKFF7tuqHrdqzVUe23dL1Mb9J15D9rfyS7AxXB0MBAP8SQ0Zkd_kbmLt-7w</recordid><startdate>20240506</startdate><enddate>20240506</enddate><creator>Yan, Zepeng</creator><creator>Sun, Wen</creator><creator>Guo, Wanli</creator><creator>Li, Biwen</creator><creator>Wen, Shiping</creator><creator>Cao, Jinde</creator><general>IEEE</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7X8</scope><orcidid>https://orcid.org/0009-0007-5291-9472</orcidid><orcidid>https://orcid.org/0000-0002-5048-0319</orcidid><orcidid>https://orcid.org/0000-0001-7070-4730</orcidid><orcidid>https://orcid.org/0000-0003-0312-7464</orcidid><orcidid>https://orcid.org/0000-0003-3133-7119</orcidid><orcidid>https://orcid.org/0000-0003-2928-6703</orcidid></search><sort><creationdate>20240506</creationdate><title>Complete Stability of Delayed Recurrent Neural Networks With New Wave-Type Activation Functions</title><author>Yan, Zepeng ; Sun, Wen ; Guo, Wanli ; Li, Biwen ; Wen, Shiping ; Cao, Jinde</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c275t-941cf0964c41eb23b38e5f0dccddf142eae972c0c7a1f58cc3258f4d31d406cd3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Artificial neural networks</topic><topic>Complete stability</topic><topic>delayed recurrent neural network (DRNN)</topic><topic>Delays</topic><topic>Neurons</topic><topic>Numerical stability</topic><topic>Stability criteria</topic><topic>Sun</topic><topic>Thermal stability</topic><topic>time-varying delay</topic><topic>wave-type activation function</topic><toplevel>online_resources</toplevel><creatorcontrib>Yan, Zepeng</creatorcontrib><creatorcontrib>Sun, Wen</creatorcontrib><creatorcontrib>Guo, Wanli</creatorcontrib><creatorcontrib>Li, Biwen</creatorcontrib><creatorcontrib>Wen, Shiping</creatorcontrib><creatorcontrib>Cao, Jinde</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998–Present</collection><collection>IEEE Xplore</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>MEDLINE - Academic</collection><jtitle>IEEE transaction on neural networks and learning systems</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Yan, Zepeng</au><au>Sun, Wen</au><au>Guo, Wanli</au><au>Li, Biwen</au><au>Wen, Shiping</au><au>Cao, Jinde</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Complete Stability of Delayed Recurrent Neural Networks With New Wave-Type Activation Functions</atitle><jtitle>IEEE transaction on neural networks and learning systems</jtitle><stitle>TNNLS</stitle><addtitle>IEEE Trans Neural Netw Learn Syst</addtitle><date>2024-05-06</date><risdate>2024</risdate><volume>PP</volume><spage>1</spage><epage>13</epage><pages>1-13</pages><issn>2162-237X</issn><eissn>2162-2388</eissn><coden>ITNNAL</coden><abstract><![CDATA[Activation functions have a significant effect on the dynamics of neural networks (NNs). This study proposes new nonmonotonic wave-type activation functions and examines the complete stability of delayed recurrent NNs (DRNNs) with these activation functions. Using the geometrical properties of the wave-type activation function and subsequent iteration scheme, sufficient conditions are provided to ensure that a DRNN with n neurons has exactly <inline-formula> <tex-math notation="LaTeX">(2m</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">+</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">3)^n</tex-math> </inline-formula> equilibria, where <inline-formula> <tex-math notation="LaTeX">(m</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">+</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">2)^n</tex-math> </inline-formula> equilibria are locally exponentially stable, the remainder <inline-formula> <tex-math notation="LaTeX">(2m</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">+</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">3)^n</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">-</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">(m</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">+</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">2)^n</tex-math> </inline-formula> equilibria are unstable, and a positive integer <inline-formula> <tex-math notation="LaTeX">m</tex-math> </inline-formula> is related to wave-type activation functions. Furthermore, the DRNN with the proposed activation function is completely stable. Compared with the previous literature, the total number of equilibria and the stable equilibria significantly increase, thereby enhancing the memory storage capacity of DRNN. Finally, several examples are presented to demonstrate our proposed results.]]></abstract><cop>United States</cop><pub>IEEE</pub><pmid>38709607</pmid><doi>10.1109/TNNLS.2024.3394854</doi><tpages>13</tpages><orcidid>https://orcid.org/0009-0007-5291-9472</orcidid><orcidid>https://orcid.org/0000-0002-5048-0319</orcidid><orcidid>https://orcid.org/0000-0001-7070-4730</orcidid><orcidid>https://orcid.org/0000-0003-0312-7464</orcidid><orcidid>https://orcid.org/0000-0003-3133-7119</orcidid><orcidid>https://orcid.org/0000-0003-2928-6703</orcidid></addata></record>
fulltext	fulltext
identifier	ISSN: 2162-237X
ispartof	IEEE transaction on neural networks and learning systems, 2024-05, Vol.PP, p.1-13
issn	2162-237X 2162-2388
language	eng
recordid	cdi_ieee_primary_10520819
source	IEEE Electronic Library (IEL) Journals
subjects	Artificial neural networks Complete stability delayed recurrent neural network (DRNN) Delays Neurons Numerical stability Stability criteria Sun Thermal stability time-varying delay wave-type activation function
title	Complete Stability of Delayed Recurrent Neural Networks With New Wave-Type Activation Functions
url	http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-24T04%3A04%3A40IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_ieee_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Complete%20Stability%20of%20Delayed%20Recurrent%20Neural%20Networks%20With%20New%20Wave-Type%20Activation%20Functions&rft.jtitle=IEEE%20transaction%20on%20neural%20networks%20and%20learning%20systems&rft.au=Yan,%20Zepeng&rft.date=2024-05-06&rft.volume=PP&rft.spage=1&rft.epage=13&rft.pages=1-13&rft.issn=2162-237X&rft.eissn=2162-2388&rft.coden=ITNNAL&rft_id=info:doi/10.1109/TNNLS.2024.3394854&rft_dat=%3Cproquest_ieee_%3E3051937817%3C/proquest_ieee_%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c275t-941cf0964c41eb23b38e5f0dccddf142eae972c0c7a1f58cc3258f4d31d406cd3%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=3051937817&rft_id=info:pmid/38709607&rft_ieee_id=10520819&rfr_iscdi=true