An Architecture for Deploying Reinforcement Learning in Industrial Environments

Industry 4.0 is driven by demands like shorter time-to-market, mass customization of products, and batch size one production. Reinforcement Learning (RL), a machine learning paradigm shown to possess a great potential in improving and surpassing human level performance in numerous complex tasks, all...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-06
Main Authors: Schäfer, Georg, Kozlica, Reuf, Wegenkittl, Stefan, Huber, Stefan
Format: Article
Language:English
Subjects:
Digital twins
Human performance
Industry 4.0
Machine learning
Task complexity
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites
container_end_page
container_issue
container_start_page
container_title arXiv.org
container_volume
creator Schäfer, Georg
Kozlica, Reuf
Wegenkittl, Stefan
Huber, Stefan
description Industry 4.0 is driven by demands like shorter time-to-market, mass customization of products, and batch size one production. Reinforcement Learning (RL), a machine learning paradigm shown to possess a great potential in improving and surpassing human level performance in numerous complex tasks, allows coping with the mentioned demands. In this paper, we present an OPC UA based Operational Technology (OT)-aware RL architecture, which extends the standard RL setting, combining it with the setting of digital twins. Moreover, we define an OPC UA information model allowing for a generalized plug-and-play like approach for exchanging the RL agent used. In conclusion, we demonstrate and evaluate the architecture, by creating a proof of concept. By means of solving a toy example, we show that this architecture can be used to determine the optimal policy using a real control system.
doi_str_mv 10.48550/arxiv.2306.01420
format article
fullrecord <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_2822562655</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2822562655</sourcerecordid><originalsourceid>FETCH-LOGICAL-a525-ac01869659cb6f33c95f76b6fa768307532ab3674fa8e33535ee514e1bcf0aa13</originalsourceid><addsrcrecordid>eNotj0FLw0AUhBdBsNT-AG8LnhN338vbJMdQqxYCBem9bNYX3RI3dZMU_fem6GmGj2GGEeJOqzQriNSDjd_-nAIqkyqdgboSC0DUSZEB3IjVMByVUmByIMKF2FVBVtF9-JHdOEWWbR_lI5-6_seHd_nKPszE8SeHUdZsY7hgH-Q2vE3DGL3t5CacfezDJTLciuvWdgOv_nUp9k-b_folqXfP23VVJ5aAEuuULkxpqHSNaRFdSW1uZmtzU6DKCcE2aPKstQUjEhIz6Yx141plrcaluP-rPcX-a-JhPBz7KYZ58QAFABkw871f2X1QdA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2822562655</pqid></control><display><type>article</type><title>An Architecture for Deploying Reinforcement Learning in Industrial Environments</title><source>Publicly Available Content Database</source><creator>Schäfer, Georg ; Kozlica, Reuf ; Wegenkittl, Stefan ; Huber, Stefan</creator><creatorcontrib>Schäfer, Georg ; Kozlica, Reuf ; Wegenkittl, Stefan ; Huber, Stefan</creatorcontrib><description>Industry 4.0 is driven by demands like shorter time-to-market, mass customization of products, and batch size one production. Reinforcement Learning (RL), a machine learning paradigm shown to possess a great potential in improving and surpassing human level performance in numerous complex tasks, allows coping with the mentioned demands. In this paper, we present an OPC UA based Operational Technology (OT)-aware RL architecture, which extends the standard RL setting, combining it with the setting of digital twins. Moreover, we define an OPC UA information model allowing for a generalized plug-and-play like approach for exchanging the RL agent used. In conclusion, we demonstrate and evaluate the architecture, by creating a proof of concept. By means of solving a toy example, we show that this architecture can be used to determine the optimal policy using a real control system.</description><identifier>EISSN: 2331-8422</identifier><identifier>DOI: 10.48550/arxiv.2306.01420</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Digital twins ; Human performance ; Industry 4.0 ; Machine learning ; Task complexity</subject><ispartof>arXiv.org, 2023-06</ispartof><rights>2023. This work is published under http://creativecommons.org/licenses/by-nc-nd/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.proquest.com/docview/2822562655?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>780,784,25753,27925,37012,44590</link.rule.ids></links><search><creatorcontrib>Schäfer, Georg</creatorcontrib><creatorcontrib>Kozlica, Reuf</creatorcontrib><creatorcontrib>Wegenkittl, Stefan</creatorcontrib><creatorcontrib>Huber, Stefan</creatorcontrib><title>An Architecture for Deploying Reinforcement Learning in Industrial Environments</title><title>arXiv.org</title><description>Industry 4.0 is driven by demands like shorter time-to-market, mass customization of products, and batch size one production. Reinforcement Learning (RL), a machine learning paradigm shown to possess a great potential in improving and surpassing human level performance in numerous complex tasks, allows coping with the mentioned demands. In this paper, we present an OPC UA based Operational Technology (OT)-aware RL architecture, which extends the standard RL setting, combining it with the setting of digital twins. Moreover, we define an OPC UA information model allowing for a generalized plug-and-play like approach for exchanging the RL agent used. In conclusion, we demonstrate and evaluate the architecture, by creating a proof of concept. By means of solving a toy example, we show that this architecture can be used to determine the optimal policy using a real control system.</description><subject>Digital twins</subject><subject>Human performance</subject><subject>Industry 4.0</subject><subject>Machine learning</subject><subject>Task complexity</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>PIMPY</sourceid><recordid>eNotj0FLw0AUhBdBsNT-AG8LnhN338vbJMdQqxYCBem9bNYX3RI3dZMU_fem6GmGj2GGEeJOqzQriNSDjd_-nAIqkyqdgboSC0DUSZEB3IjVMByVUmByIMKF2FVBVtF9-JHdOEWWbR_lI5-6_seHd_nKPszE8SeHUdZsY7hgH-Q2vE3DGL3t5CacfezDJTLciuvWdgOv_nUp9k-b_folqXfP23VVJ5aAEuuULkxpqHSNaRFdSW1uZmtzU6DKCcE2aPKstQUjEhIz6Yx141plrcaluP-rPcX-a-JhPBz7KYZ58QAFABkw871f2X1QdA</recordid><startdate>20230602</startdate><enddate>20230602</enddate><creator>Schäfer, Georg</creator><creator>Kozlica, Reuf</creator><creator>Wegenkittl, Stefan</creator><creator>Huber, Stefan</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20230602</creationdate><title>An Architecture for Deploying Reinforcement Learning in Industrial Environments</title><author>Schäfer, Georg ; Kozlica, Reuf ; Wegenkittl, Stefan ; Huber, Stefan</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a525-ac01869659cb6f33c95f76b6fa768307532ab3674fa8e33535ee514e1bcf0aa13</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Digital twins</topic><topic>Human performance</topic><topic>Industry 4.0</topic><topic>Machine learning</topic><topic>Task complexity</topic><toplevel>online_resources</toplevel><creatorcontrib>Schäfer, Georg</creatorcontrib><creatorcontrib>Kozlica, Reuf</creatorcontrib><creatorcontrib>Wegenkittl, Stefan</creatorcontrib><creatorcontrib>Huber, Stefan</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering collection</collection><jtitle>arXiv.org</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Schäfer, Georg</au><au>Kozlica, Reuf</au><au>Wegenkittl, Stefan</au><au>Huber, Stefan</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>An Architecture for Deploying Reinforcement Learning in Industrial Environments</atitle><jtitle>arXiv.org</jtitle><date>2023-06-02</date><risdate>2023</risdate><eissn>2331-8422</eissn><abstract>Industry 4.0 is driven by demands like shorter time-to-market, mass customization of products, and batch size one production. Reinforcement Learning (RL), a machine learning paradigm shown to possess a great potential in improving and surpassing human level performance in numerous complex tasks, allows coping with the mentioned demands. In this paper, we present an OPC UA based Operational Technology (OT)-aware RL architecture, which extends the standard RL setting, combining it with the setting of digital twins. Moreover, we define an OPC UA information model allowing for a generalized plug-and-play like approach for exchanging the RL agent used. In conclusion, we demonstrate and evaluate the architecture, by creating a proof of concept. By means of solving a toy example, we show that this architecture can be used to determine the optimal policy using a real control system.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><doi>10.48550/arxiv.2306.01420</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier EISSN: 2331-8422
ispartof arXiv.org, 2023-06
issn 2331-8422
language eng
recordid cdi_proquest_journals_2822562655
source Publicly Available Content Database
subjects Digital twins
Human performance
Industry 4.0
Machine learning
Task complexity
title An Architecture for Deploying Reinforcement Learning in Industrial Environments
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-28T19%3A38%3A40IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=An%20Architecture%20for%20Deploying%20Reinforcement%20Learning%20in%20Industrial%20Environments&rft.jtitle=arXiv.org&rft.au=Sch%C3%A4fer,%20Georg&rft.date=2023-06-02&rft.eissn=2331-8422&rft_id=info:doi/10.48550/arxiv.2306.01420&rft_dat=%3Cproquest%3E2822562655%3C/proquest%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-a525-ac01869659cb6f33c95f76b6fa768307532ab3674fa8e33535ee514e1bcf0aa13%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2822562655&rft_id=info:pmid/&rfr_iscdi=true