Teacher feedback to scaffold and refine demonstrated motion primitives on a mobile robot

• Published in: Robotics and autonomous systems 2011-03, Vol.59 (3), p.243-255
• Main Authors: Argall, Brenna D., Browning, Brett, Veloso, Manuela M.
• Format: Article
• Language: English
• Subjects: Algorithms; Control systems; Control theory; Demonstration learning; Feedback; Policies; Policy reuse; Robot control; Robot motion control; Robots; Tasks; Teacher feedback
• ISSN: 0921-8890; 1872-793X
• DOI: 10.1016/j.robot.2010.11.004

Task demonstration is an effective technique for developing robot motion control policies. As tasks become more complex, however, demonstration can become more difficult. In this work, we introduce an algorithm that uses corrective human feedback to build a policy able to perform a novel task, by combining simpler policies learned from demonstration. While some demonstration-based learning approaches do adapt policies with execution experience, few provide corrections within low-level motion control domains or to enable the linking of multiple of demonstrated policies. Here we introduce Feedback for Policy Scaffolding (FPS) as an algorithm that first evaluates and corrects the execution of motion primitive policies learned from demonstration. The algorithm next corrects and enables the execution of a more complex task constructed from these primitives. Key advantages of building a policy from demonstrated primitives is the potential for primitive policy reuse within multiple complex policies and the faster development of these policies, in addition to the development of complex policies for which full demonstration is difficult. Policy reuse under our algorithm is assisted by human teacher feedback, which also contributes to the improvement of policy performance. Within a simulated robot motion control domain we validate that, using FPS, a policy for a novel task is successfully built from motion primitives learned from demonstration. We show feedback to both aid and enable policy development, improving policy performance in success, speed and efficiency. ► Approach for mobile robot motion control from demonstration, feedback and scaffolding. ► Motion primitives are demonstrated, and scaffolded into more complex behaviors. ► Corrective feedback assists in policy scaffolding, and thus enables new behavior. ► Feedback also shown to improve both primitive and complex policy performance. ► Empirical validation within a simulated robot motion control domain.