Toward Deep Universal Sketch Perceptual Grouper

Human free-hand sketches provide the useful data for studying human perceptual grouping, where the grouping principles such as the Gestalt laws of grouping are naturally in play during both the perception and sketching stages. In this paper, we make the first attempt to develop a universal sketch pe...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on image processing 2019-07, Vol.28 (7), p.3219-3231
Main Authors: Ke Li, Kaiyue Pang, Yi-Zhe Song, Tao Xiang, Hospedales, Timothy M., Honggang Zhang
Format: Article
Language:English
Subjects:
Analytical models
Data models
dataset
deep grouping model
Image annotation
Image management
Image retrieval
Image segmentation
Semantics
Sketch perceptual grouping
Sketches
Task analysis
Training
universal grouper
Visualization
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Human free-hand sketches provide the useful data for studying human perceptual grouping, where the grouping principles such as the Gestalt laws of grouping are naturally in play during both the perception and sketching stages. In this paper, we make the first attempt to develop a universal sketch perceptual grouper. That is, a grouper that can be applied to sketches of any category created with any drawing style and ability, to group constituent strokes/segments into semantically meaningful object parts. The first obstacle to achieving this goal is the lack of large-scale datasets with grouping annotation. To overcome this, we contribute the largest sketch perceptual grouping dataset to date, consisting of 20 000 unique sketches evenly distributed over 25 object categories. Furthermore, we propose a novel deep perceptual grouping model learned with both generative and discriminative losses. The generative loss improves the generalization ability of the model, while the discriminative loss guarantees both local and global grouping consistency. Extensive experiments demonstrate that the proposed grouper significantly outperforms the state-of-the-art competitors. In addition, we show that our grouper is useful for a number of sketch analysis tasks, including sketch semantic segmentation, synthesis, and fine-grained sketch-based image retrieval.
ISSN:1057-7149
1941-0042
DOI:10.1109/TIP.2019.2895155