Utilization of a reinforcement learning algorithm for the accurate alignment of a robotic arm in a complete soft fabric shoe tongues automation process

• Published in: Journal of manufacturing systems 2020-07, Vol.56, p.501-513
• Main Authors: Tsai, Yu-Ting, Lee, Chien-Hui, Liu, Tao-Ying, Chang, Tien-Jan, Wang, Chun-Sheng, Pawar, S.J., Huang, Pei-Hsing, Huang, Jin-H.
• Format: Article
• Language: English
• Subjects: Artificial intelligence; Cyber-physical system; Reinforcement learning; Shoemaking automation
• ISSN: 0278-6125
• DOI: 10.1016/j.jmsy.2020.07.001

•A novel production line with a cyber-physical artificial intelligence system is presented for the complete manufacturing of shoe tongues.•The production line system based on reinforcement learning is proven to be capable of achieving optimum product quality.•The deep Q-learning framework for the visual feedback and robotic arm control ensures the manufacturing precision of shoe tongues. As usher in Industry 4.0, there has been much interest in the development and research that combine artificial intelligence with automation. The control and operation of equipment in a traditional automated shoemaking production line require a preliminary subjective judgment of relevant manufacturing processes, to determine the exact procedure and corresponding control settings. However, with the manual control setting, it is difficult to achieve an accurate quality assessment of an automated process characterized by high uncertainty and intricacy. It is challenging to replace handicrafts and the versatility of manual product customization with automation techniques. Hence, the current study has developed an automatic production line with a cyber-physical system artificial intelligence (CPS-AI) architecture for the complete manufacturing of soft fabric shoe tongues. The Deep-Q reinforcement learning (RL) method is proposed as a means of achieving better control over the manufacturing process, while the convolutional and long short-term memory artificial neural network (CNN + LSTM) is developed to enhance action speed. This technology allows a robotic arm to learn the specific image feature points of a shoe tongue through repeated training to improve its manufacturing accuracy. For validation, different parameters of the network architecture are tested, and the test convergence accuracy was found to be as high as 95.9 %. During its actual implementation, the production line completed 509 finished products, of which 349 products were acceptable due to the anticipated measurement error. This showed that the production line system was capable of achieving optimum product accuracy and quality with respect to the performance of repeated computations, parameter updates, and action evaluations.