Intelligent 3D Perception System for Semantic Description and Dynamic Interaction

This work proposes a novel semantic perception system based on computer vision and machine learning techniques. The main goal is to identify objects in the environment and extract their characteristics, allowing a dynamic interaction with the environment. The system is composed of a GPU processing s...

Full description

Saved in:
Bibliographic Details
Published in:Sensors (Basel, Switzerland) Switzerland), 2019-09, Vol.19 (17), p.3764
Main Authors: Teixeira, Marco, Nogueira, Rafael, Dalmedico, Nicolas, Santos, Higor, Arruda, Lucia, Neves-Jr, Flavio, Pipa, Daniel, Ramos, Julio, Oliveira, Andre
Format: Article
Language:English
Subjects:
Acceleration
Computer vision
Decomposition
Embedded systems
Pattern recognition
perception
PointCloud
prediction and object recognition
Robotics
Robots
Semantics
Sensors
Space perception
Surveying
Syntax
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This work proposes a novel semantic perception system based on computer vision and machine learning techniques. The main goal is to identify objects in the environment and extract their characteristics, allowing a dynamic interaction with the environment. The system is composed of a GPU processing source and a 3D vision sensor that provides RGB image and PointCloud data. The perception system is structured in three steps: Lexical Analysis, Syntax Analysis and finally an Analysis of Anticipation. The Lexical Analysis detects the actual position of the objects (or tokens) in the environment, through the combination of RGB image and PointCloud, surveying their characteristics. All information extracted from the tokens will be used to retrieve relevant features such as object velocity, acceleration and direction during the Syntax Analysis step. The anticipation step predicts future behaviors for these dynamic objects, promoting an interaction with them in terms of collisions, pull, and push actions. As a result, the proposed perception source can assign relevant information to mobile robots, not only about distances as traditional sensors, but about other environment characteristics and object behaviors. This novel perception source introduces a new class of skills to mobile robots. Experimental results obtained with a real robot are presented, showing the proposed perception source efficacy and potential.
ISSN:1424-8220
1424-8220
DOI:10.3390/s19173764