Training Microrobots via Reinforcement Learning and a Novel Coding Method

• Main Authors: Jebellat, Iman, Pishkenari, Hossein Nejat, Jebellat, Ehsan
• Format: Conference Proceeding
• Language: English
• Subjects: Binary Coding; Codes; Encoding; Low Reynolds; Mechatronics; Microsurgery; Microswimmers; Propulsion; Propulsion Cycle; Q-learning; Reinforcement Learning; Smart Microrobots; Training
• ISSN: 2572-6889
• DOI: 10.1109/ICRoM54204.2021.9663463

Microswimmers have promising applications in different fields, especially in biomedical branches, like microsurgery, therapeutic delivery, and driving sperm cells. Many researchers have proposed different synthetic microrobots with various mechanisms and effective propulsion strategies. Based on the Scallop Theorem, only special sequences of movements cause an effective swimming, so finding these sequences are important. Researchers have been striving to improve microrobots' propulsion strategies and find new ways to reach optimal propulsion tactics. One of the excellent approaches is using Artificial Intelligence branches, like Reinforcement Learning, to train microrobots. In this study, we introduce a novel coding method called "Binary Coding," which can be utilized in applying Reinforcement Learning algorithms to different microrobots. In this work, we train linear microrobots consisting of spheres and extensible connecting rods between them via Binary Coding and two Reinforcement Learning algorithms, Q-Learning and Expected Value SARSA. For this purpose, we consider several episodes, each of which has sufficient learning steps. By selecting appropriate values for learning parameters, the smart microrobots learn the optimal propulsion cycle (sequence of movements) by themselves and generate a large net displacement in the final episode.