RGB-D-Based Object Recognition Using Multimodal Convolutional Neural Networks: A Survey

Object recognition in real-world environments is one of the fundamental and key tasks in computer vision and robotics communities. With the advanced sensing technologies and low-cost depth sensors, the high-quality RGB and depth images can be recorded synchronously, and the object recognition perfor...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2019, Vol.7, p.43110-43136
Main Authors: Gao, Mingliang, Jiang, Jun, Zou, Guofeng, John, Vijay, Liu, Zheng
Format: Article
Language:English
Subjects:
Artificial neural networks
Computer vision
Convolutional neural network
Datasets
Deep learning
Feature extraction
Histograms
Image color analysis
Image processing
Image quality
Machine learning
Medical imaging
Multimedia
multimodal fusion
Neural networks
Object recognition
Performance evaluation
RGB-D
Robotics
Speech recognition
State-of-the-art reviews
survey
Task analysis
Three-dimensional displays
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Object recognition in real-world environments is one of the fundamental and key tasks in computer vision and robotics communities. With the advanced sensing technologies and low-cost depth sensors, the high-quality RGB and depth images can be recorded synchronously, and the object recognition performance can be improved by jointly exploiting them. RGB-D-based object recognition has evolved from early methods that using hand-crafted representations to the current state-of-the-art deep learning-based methods. With the undeniable success of deep learning, especially convolutional neural networks (CNNs) in the visual domain, the natural progression of deep learning research points to problems involving larger and more complex multimodal data. In this paper, we provide a comprehensive survey of recent multimodal CNNs (MMCNNs)-based approaches that have demonstrated significant improvements over previous methods. We highlight two key issues, namely, training data deficiency and multimodal fusion. In addition, we summarize and discuss the publicly available RGB-D object recognition datasets and present a comparative performance evaluation of the proposed methods on these benchmark datasets. Finally, we identify promising avenues of research in this rapidly evolving field. This survey will not only enable researchers to get a good overview of the state-of-the-art methods for RGB-D-based object recognition but also provide a reference for other multimodal machine learning applications, e.g., multimodal medical image fusion, audio-visual speech recognition, and multimedia retrieval and generation.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2019.2907071