Visual Social Relationship Recognition

Social relationships form the basis of social structure of humans. Developing computational models to understand social relationships from visual data is essential for building intelligent machines that can better interact with humans in a social environment. In this work, we study the problem of vi...

Full description

Saved in:
Bibliographic Details
Published in:International journal of computer vision 2020-06, Vol.128 (6), p.1750-1764
Main Authors: Li, Junnan, Wong, Yongkang, Zhao, Qi, Kankanhalli, Mohan S.
Format: Article
Language:English
Subjects:
Annotations
Artificial Intelligence
Computer Imaging
Computer Science
Image Processing and Computer Vision
Object recognition
Pattern Recognition
Pattern Recognition and Graphics
Vision
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Social relationships form the basis of social structure of humans. Developing computational models to understand social relationships from visual data is essential for building intelligent machines that can better interact with humans in a social environment. In this work, we study the problem of visual social relationship recognition in images. We propose a dual-glance model for social relationship recognition, where the first glance fixates at the person of interest and the second glance deploys attention mechanism to exploit contextual cues. To enable this study, we curated a large scale People in Social Context dataset, which comprises of 23,311 images and 79,244 person pairs with annotated social relationships. Since visually identifying social relationship bears certain degree of uncertainty, we further propose an adaptive focal loss to leverage the ambiguous annotations for more effective learning. We conduct extensive experiments to quantitatively and qualitatively demonstrate the efficacy of our proposed method, which yields state-of-the-art performance on social relationship recognition.
ISSN:0920-5691
1573-1405
DOI:10.1007/s11263-020-01295-1