DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots

We present DemoStart, a novel auto-curriculum reinforcement learning method capable of learning complex manipulation behaviors on an arm equipped with a three-fingered robotic hand, from only a sparse reward and a handful of demonstrations in simulation. Learning from simulation drastically reduces...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-09
Main Authors: Bauza, Maria, Chen, Jose Enrique, Dalibard, Valentin, Gileadi, Nimrod, Hafner, Roland, Martins, Murilo F, Moore, Joss, Pevceviciute, Rugile, Laurens, Antoine, Rao, Dushyant, Zambelli, Martina, Riedmiller, Martin, Scholz, Jon, Bousmalis, Konstantinos, Nori, Francesco, Heess, Nicolas
Format: Article
Language:English
Subjects:
Curricula
End effectors
Policies
Robot learning
Robots
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites
container_end_page
container_issue
container_start_page
container_title arXiv.org
container_volume
creator Bauza, Maria
Chen, Jose Enrique
Dalibard, Valentin
Gileadi, Nimrod
Hafner, Roland
Martins, Murilo F
Moore, Joss
Pevceviciute, Rugile
Laurens, Antoine
Rao, Dushyant
Zambelli, Martina
Riedmiller, Martin
Scholz, Jon
Bousmalis, Konstantinos
Nori, Francesco
Heess, Nicolas
description We present DemoStart, a novel auto-curriculum reinforcement learning method capable of learning complex manipulation behaviors on an arm equipped with a three-fingered robotic hand, from only a sparse reward and a handful of demonstrations in simulation. Learning from simulation drastically reduces the development cycle of behavior generation, and domain randomization techniques are leveraged to achieve successful zero-shot sim-to-real transfer. Transferred policies are learned directly from raw pixels from multiple cameras and robot proprioception. Our approach outperforms policies learned from demonstrations on the real robot and requires 100 times fewer demonstrations, collected in simulation. More details and videos in https://sites.google.com/view/demostart.
format article
fullrecord <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_3103020559</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>3103020559</sourcerecordid><originalsourceid>FETCH-proquest_journals_31030205593</originalsourceid><addsrcrecordid>eNqNiskKwjAURYMgWLT_EHAdSBPjtHXAva4tsaaakqG-vODvW8EPcHUP95wRKYSUFVsvhJiQMqWOcy6WK6GULMh1b3w8owbc0i-GhKDRxsCcuVOdMbImA9gmu-yp7ntnhx8jTdazQYLRjr4tPqnPDi1rbXgYGBKIt4hpRsatdsmUv52S-fFw2Z1YD_GVTcK6ixnCoGpZcckFV2oj_6s-s4JEvw</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3103020559</pqid></control><display><type>article</type><title>DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots</title><source>Publicly Available Content Database (Proquest) (PQ_SDU_P3)</source><creator>Bauza, Maria ; Chen, Jose Enrique ; Dalibard, Valentin ; Gileadi, Nimrod ; Hafner, Roland ; Martins, Murilo F ; Moore, Joss ; Pevceviciute, Rugile ; Laurens, Antoine ; Rao, Dushyant ; Zambelli, Martina ; Riedmiller, Martin ; Scholz, Jon ; Bousmalis, Konstantinos ; Nori, Francesco ; Heess, Nicolas</creator><creatorcontrib>Bauza, Maria ; Chen, Jose Enrique ; Dalibard, Valentin ; Gileadi, Nimrod ; Hafner, Roland ; Martins, Murilo F ; Moore, Joss ; Pevceviciute, Rugile ; Laurens, Antoine ; Rao, Dushyant ; Zambelli, Martina ; Riedmiller, Martin ; Scholz, Jon ; Bousmalis, Konstantinos ; Nori, Francesco ; Heess, Nicolas</creatorcontrib><description>We present DemoStart, a novel auto-curriculum reinforcement learning method capable of learning complex manipulation behaviors on an arm equipped with a three-fingered robotic hand, from only a sparse reward and a handful of demonstrations in simulation. Learning from simulation drastically reduces the development cycle of behavior generation, and domain randomization techniques are leveraged to achieve successful zero-shot sim-to-real transfer. Transferred policies are learned directly from raw pixels from multiple cameras and robot proprioception. Our approach outperforms policies learned from demonstrations on the real robot and requires 100 times fewer demonstrations, collected in simulation. More details and videos in https://sites.google.com/view/demostart.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Curricula ; End effectors ; Policies ; Robot learning ; Robots</subject><ispartof>arXiv.org, 2024-09</ispartof><rights>2024. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.proquest.com/docview/3103020559?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>780,784,25753,37012,44590</link.rule.ids></links><search><creatorcontrib>Bauza, Maria</creatorcontrib><creatorcontrib>Chen, Jose Enrique</creatorcontrib><creatorcontrib>Dalibard, Valentin</creatorcontrib><creatorcontrib>Gileadi, Nimrod</creatorcontrib><creatorcontrib>Hafner, Roland</creatorcontrib><creatorcontrib>Martins, Murilo F</creatorcontrib><creatorcontrib>Moore, Joss</creatorcontrib><creatorcontrib>Pevceviciute, Rugile</creatorcontrib><creatorcontrib>Laurens, Antoine</creatorcontrib><creatorcontrib>Rao, Dushyant</creatorcontrib><creatorcontrib>Zambelli, Martina</creatorcontrib><creatorcontrib>Riedmiller, Martin</creatorcontrib><creatorcontrib>Scholz, Jon</creatorcontrib><creatorcontrib>Bousmalis, Konstantinos</creatorcontrib><creatorcontrib>Nori, Francesco</creatorcontrib><creatorcontrib>Heess, Nicolas</creatorcontrib><title>DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots</title><title>arXiv.org</title><description>We present DemoStart, a novel auto-curriculum reinforcement learning method capable of learning complex manipulation behaviors on an arm equipped with a three-fingered robotic hand, from only a sparse reward and a handful of demonstrations in simulation. Learning from simulation drastically reduces the development cycle of behavior generation, and domain randomization techniques are leveraged to achieve successful zero-shot sim-to-real transfer. Transferred policies are learned directly from raw pixels from multiple cameras and robot proprioception. Our approach outperforms policies learned from demonstrations on the real robot and requires 100 times fewer demonstrations, collected in simulation. More details and videos in https://sites.google.com/view/demostart.</description><subject>Curricula</subject><subject>End effectors</subject><subject>Policies</subject><subject>Robot learning</subject><subject>Robots</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>PIMPY</sourceid><recordid>eNqNiskKwjAURYMgWLT_EHAdSBPjtHXAva4tsaaakqG-vODvW8EPcHUP95wRKYSUFVsvhJiQMqWOcy6WK6GULMh1b3w8owbc0i-GhKDRxsCcuVOdMbImA9gmu-yp7ntnhx8jTdazQYLRjr4tPqnPDi1rbXgYGBKIt4hpRsatdsmUv52S-fFw2Z1YD_GVTcK6ixnCoGpZcckFV2oj_6s-s4JEvw</recordid><startdate>20240912</startdate><enddate>20240912</enddate><creator>Bauza, Maria</creator><creator>Chen, Jose Enrique</creator><creator>Dalibard, Valentin</creator><creator>Gileadi, Nimrod</creator><creator>Hafner, Roland</creator><creator>Martins, Murilo F</creator><creator>Moore, Joss</creator><creator>Pevceviciute, Rugile</creator><creator>Laurens, Antoine</creator><creator>Rao, Dushyant</creator><creator>Zambelli, Martina</creator><creator>Riedmiller, Martin</creator><creator>Scholz, Jon</creator><creator>Bousmalis, Konstantinos</creator><creator>Nori, Francesco</creator><creator>Heess, Nicolas</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20240912</creationdate><title>DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots</title><author>Bauza, Maria ; Chen, Jose Enrique ; Dalibard, Valentin ; Gileadi, Nimrod ; Hafner, Roland ; Martins, Murilo F ; Moore, Joss ; Pevceviciute, Rugile ; Laurens, Antoine ; Rao, Dushyant ; Zambelli, Martina ; Riedmiller, Martin ; Scholz, Jon ; Bousmalis, Konstantinos ; Nori, Francesco ; Heess, Nicolas</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_31030205593</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Curricula</topic><topic>End effectors</topic><topic>Policies</topic><topic>Robot learning</topic><topic>Robots</topic><toplevel>online_resources</toplevel><creatorcontrib>Bauza, Maria</creatorcontrib><creatorcontrib>Chen, Jose Enrique</creatorcontrib><creatorcontrib>Dalibard, Valentin</creatorcontrib><creatorcontrib>Gileadi, Nimrod</creatorcontrib><creatorcontrib>Hafner, Roland</creatorcontrib><creatorcontrib>Martins, Murilo F</creatorcontrib><creatorcontrib>Moore, Joss</creatorcontrib><creatorcontrib>Pevceviciute, Rugile</creatorcontrib><creatorcontrib>Laurens, Antoine</creatorcontrib><creatorcontrib>Rao, Dushyant</creatorcontrib><creatorcontrib>Zambelli, Martina</creatorcontrib><creatorcontrib>Riedmiller, Martin</creatorcontrib><creatorcontrib>Scholz, Jon</creatorcontrib><creatorcontrib>Bousmalis, Konstantinos</creatorcontrib><creatorcontrib>Nori, Francesco</creatorcontrib><creatorcontrib>Heess, Nicolas</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>ProQuest Central Essentials</collection><collection>AUTh Library subscriptions: ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database (Proquest) (PQ_SDU_P3)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Bauza, Maria</au><au>Chen, Jose Enrique</au><au>Dalibard, Valentin</au><au>Gileadi, Nimrod</au><au>Hafner, Roland</au><au>Martins, Murilo F</au><au>Moore, Joss</au><au>Pevceviciute, Rugile</au><au>Laurens, Antoine</au><au>Rao, Dushyant</au><au>Zambelli, Martina</au><au>Riedmiller, Martin</au><au>Scholz, Jon</au><au>Bousmalis, Konstantinos</au><au>Nori, Francesco</au><au>Heess, Nicolas</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots</atitle><jtitle>arXiv.org</jtitle><date>2024-09-12</date><risdate>2024</risdate><eissn>2331-8422</eissn><abstract>We present DemoStart, a novel auto-curriculum reinforcement learning method capable of learning complex manipulation behaviors on an arm equipped with a three-fingered robotic hand, from only a sparse reward and a handful of demonstrations in simulation. Learning from simulation drastically reduces the development cycle of behavior generation, and domain randomization techniques are leveraged to achieve successful zero-shot sim-to-real transfer. Transferred policies are learned directly from raw pixels from multiple cameras and robot proprioception. Our approach outperforms policies learned from demonstrations on the real robot and requires 100 times fewer demonstrations, collected in simulation. More details and videos in https://sites.google.com/view/demostart.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier EISSN: 2331-8422
ispartof arXiv.org, 2024-09
issn 2331-8422
language eng
recordid cdi_proquest_journals_3103020559
source Publicly Available Content Database (Proquest) (PQ_SDU_P3)
subjects Curricula
End effectors
Policies
Robot learning
Robots
title DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-02T05%3A11%3A28IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=DemoStart:%20Demonstration-led%20auto-curriculum%20applied%20to%20sim-to-real%20with%20multi-fingered%20robots&rft.jtitle=arXiv.org&rft.au=Bauza,%20Maria&rft.date=2024-09-12&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E3103020559%3C/proquest%3E%3Cgrp_id%3Ecdi_FETCH-proquest_journals_31030205593%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=3103020559&rft_id=info:pmid/&rfr_iscdi=true