Visual search and recognition for robot task execution and monitoring

Visual search of relevant targets in the environment is a crucial robot skill. We propose a preliminary framework for the execution monitor of a robot task, taking care of the robot attitude to visually searching the environment for targets involved in the task. Visual search is also relevant to rec...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2019-02
Main Authors: Mauro, Lorenzo, Puja, Francesco, Grazioso, Simone, Valsamis Ntouskos, Sanzari, Marta, Alati, Edoardo, Pirri, Fiora
Format: Article
Language:English
Subjects:
Artificial neural networks
Autonomy
Monitoring
Object recognition
Robots
Searching
Visual tasks
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Visual search of relevant targets in the environment is a crucial robot skill. We propose a preliminary framework for the execution monitor of a robot task, taking care of the robot attitude to visually searching the environment for targets involved in the task. Visual search is also relevant to recover from a failure. The framework exploits deep reinforcement learning to acquire a "common sense" scene structure and it takes advantage of a deep convolutional network to detect objects and relevant relations holding between them. The framework builds on these methods to introduce a vision-based execution monitoring, which uses classical planning as a backbone for task execution. Experiments show that with the proposed vision-based execution monitor the robot can complete simple tasks and can recover from failures in autonomy.
ISSN:2331-8422
DOI:10.48550/arxiv.1902.02870