RL-RRT: Kinodynamic Motion Planning via Learning Reachability Estimators From RL Policies

• Published in: IEEE robotics and automation letters 2019-10, Vol.4 (4), p.4298-4305
• Main Authors: Chiang, Hao-Tien Lewis, Hsu, Jasmine, Fiser, Marek, Tapia, Lydia, Faust, Aleksandra
• Format: Article
• Language: English
• Subjects: Collision avoidance; deep learning in robotics and automation; Laser radar; learning and adaptive systems; Machine learning; Motion and path planning; Motion planning; Neural networks; Noise measurement; Planning; Robot sensing systems; Robots; Sampling; Steering; Training
• ISSN: 2377-3766; 2377-3766
• DOI: 10.1109/LRA.2019.2931199

This letter addresses two challenges facing samplingbased kinodynamic motion planning: a way to identify good candidate states for local transitions and the subsequent computationally intractable steering between these candidate states. Through the combination of sampling-based planning, a Rapidly Exploring Randomized Tree (RRT) and an efficient kinodynamic motion planner through machine learning, we propose an efficient solution to long-range planning for kinodynamic motion planning. First, we use deep reinforcement learning to learn an obstacle-avoiding policy that maps a robot's sensor observations to actions, which is used as a local planner during planning and as a controller during execution. Second, we train a reachability estimator in a supervised manner, which predicts the RL policy's time to reach a state in the presence of obstacles. Lastly, we introduce RL-RRT that uses the RL policy as a local planner, and the reachability estimator as the distance function to bias tree-growth towards promising regions. We evaluate our method on three kinodynamic systems, including physical robot experiments. Results across all three robots tested indicate that RL-RRT outperforms state of the art kinodynamic planners in efficiency, and also provides a shorter path finish time than a steering function free method. The learned local planner policy and accompanying reachability estimator demonstrate transferability to the previously unseen experimental environments, making RL-RRT fast because the expensive computations are replaced with simple neural network inference.