A saliency-based reinforcement learning approach for a UAV to avoid flying obstacles

• Published in: Robotics and autonomous systems 2018-02, Vol.100, p.108-118
• Main Authors: Ma, Zhaowei, Wang, Chang, Niu, Yifeng, Wang, Xiangke, Shen, Lincheng
• Format: Article
• Language: English
• Subjects: Convolution neural networks based saliency detection; Flying obstacle avoidance; Reinforcement learning; UAV
• ISSN: 0921-8890; 1872-793X
• DOI: 10.1016/j.robot.2017.10.009

Obstacle avoidance is a necessary behavior to guarantee the safety of an unmanned aerial vehicle (UAV). However, it is a challenge for the UAV to detect and avoid high-speed flying obstacles such as other UAVs or birds. In this paper, we propose a generic framework that integrates an autonomous obstacle detection module and a reinforcement learning (RL) module to develop reactive obstacle avoidance behavior for a UAV. In the obstacle detection module, we design a saliency detection algorithm using deep convolution neural networks (CNNs) to extract monocular visual cues. The algorithm imitates human’s visual detection system, and it can accurately estimate the location of obstacles in the field of view (FOV). The RL module uses an actor–critic structure that chooses the RBF neural network to approximate the value function and control policy in continuous state and action spaces. We have tested the effectiveness of the proposed learning framework in a semi-physical experiment. The results show that the proposed saliency detection algorithm performs better than state-of-the-art, and the RL algorithm can learn the avoidance behavior from the manual experiences. •We propose a novel framework in which a UAV can learn to avoid flying obstacles.•We propose an improved saliency detection method based on convolution neural networks.•We propose an actor–critic reinforcement learning algorithm to control UAV in continuous spaces.