Sketch-R2CNN: An RNN-Rasterization-CNN Architecture for Vector Sketch Recognition

Sketches in existing large-scale datasets like the recent QuickDraw collection are often stored in a vector format, with strokes consisting of sequentially sampled points. However, most existing sketch recognition methods rasterize vector sketches as binary images and then adopt image classification...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on visualization and computer graphics 2021-09, Vol.27 (9), p.3745-3754
Main Authors: Li, Lei, Zou, Changqing, Zheng, Youyi, Su, Qingkun, Fu, Hongbo, Tai, Chiew-Lan
Format: Article
Language:English
Subjects:
CNN
Computer architecture
Datasets
Feature extraction
Feature maps
Format
Freehand sketching
Image classification
Image recognition
Modules
Network architecture
neural rasterization
object classification
Object recognition
QuickDraw
Recurrent neural networks
RNN
Sketches
Task analysis
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Sketches in existing large-scale datasets like the recent QuickDraw collection are often stored in a vector format, with strokes consisting of sequentially sampled points. However, most existing sketch recognition methods rasterize vector sketches as binary images and then adopt image classification techniques. In this article, we propose a novel end-to-end single-branch network architecture RNN-Rasterization-CNN ( Sketch-R2CNN for short) to fully leverage the vector format of sketches for recognition. Sketch-R2CNN takes a vector sketch as input and uses an RNN for extracting per-point features in the vector space. We then develop a neural line rasterization module to convert the vector sketch and the per-point features to multi-channel point feature maps, which are subsequently fed to a CNN for extracting convolutional features in the pixel space. Our neural line rasterization module is designed in a differentiable way for end-to-end learning. We perform experiments on existing large-scale sketch recognition datasets and show that the RNN-Rasterization design brings consistent improvement over CNN baselines and that Sketch-R2CNN substantially outperforms the state-of-the-art methods.
ISSN:1077-2626
1941-0506
DOI:10.1109/TVCG.2020.2987626