Domain Stylization: A Fast Covariance Matching Framework Towards Domain Adaptation

Generating computer graphics (CG) rendered synthetic images has been widely used to create simulation environments for robotics/autonomous driving and generate labeled data. Yet, the problem of training models purely with synthetic data remains challenging due to the considerable domain gaps caused...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2021-07, Vol.43 (7), p.2360-2372
Main Authors: Dundar, Aysegul, Liu, Ming-Yu, Yu, Zhiding, Wang, Ting-Chun, Zedlewski, John, Kautz, Jan
Format: Article
Language:English
Subjects:
Adaptation
Adaptation models
Computer graphics
Covariance
Data models
Domain adaptation
Domains
Gallium nitride
Image segmentation
image stylization
Matching
object detection
Robotics
semantic segmentation
Semantics
Task analysis
Training
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Generating computer graphics (CG) rendered synthetic images has been widely used to create simulation environments for robotics/autonomous driving and generate labeled data. Yet, the problem of training models purely with synthetic data remains challenging due to the considerable domain gaps caused by current limitations on rendering. In this paper, we propose a simple yet effective domain adaptation framework towards closing such gap at image level. Unlike many GAN-based approaches, our method aims to match the covariance of the universal feature embeddings across domains, making the adaptation a fast, convenient step and avoiding the need for potentially difficult GAN training. To align domains more precisely, we further propose a conditional covariance matching framework which iteratively estimates semantic segmentation regions and conditionally matches the class-wise feature covariance given the segmentation regions. We demonstrate that both tasks can mutually refine and considerably improve each other, leading to state-of-the-art domain adaptation results. Extensive experiments under multiple synthetic-to-real settings show that our approach exceeds the performance of latest domain adaptation approaches. In addition, we offer a quantitative analysis where our framework shows considerable reduction in Frechet Inception distance between source and target domains, demonstrating the effectiveness of this work in bridging the synthetic-to-real domain gap.
ISSN:0162-8828
1939-3539
2160-9292
DOI:10.1109/TPAMI.2020.2969421