Style transfer with target feature palette and attention coloring

Style transfer has attracted a lot of attentions, as it can change a given image into one with splendid artistic styles while preserving the image structure. However, conventional approaches easily lose image details and tend to produce unpleasant artifacts during style transfer. In this paper, to s...

Full description

Saved in:
Bibliographic Details
Published in:Multimedia tools and applications 2022-11, Vol.81 (27), p.39675-39694
Main Authors: Ha, Suhyeon, Kim, Guisik, Kwon, Junseok
Format: Article
Language:English
Subjects:
Ablation
Cluster analysis
Clustering
Coloring
Computer Communication Networks
Computer Science
Data Structures and Information Theory
Deep learning
Methods
Modules
Multimedia
Multimedia Information Systems
Neural networks
Optimization
Special Purpose and Application-Based Systems
Vector quantization
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Style transfer has attracted a lot of attentions, as it can change a given image into one with splendid artistic styles while preserving the image structure. However, conventional approaches easily lose image details and tend to produce unpleasant artifacts during style transfer. In this paper, to solve these problems, a novel artistic stylization method with target feature palettes is proposed, which can transfer key features accurately. Specifically, our method contains two modules, namely feature palette composition (FPC) and attention coloring (AC) modules. The FPC module captures representative features based on K-means clustering and produces a feature target palette. The following AC module calculates attention maps between content and style images, and transfers colors and patterns based on the attention map and the target palette. These modules enable the proposed stylization to focus on key features and generate plausibly transferred images. Thus, the contributions of the proposed method are to propose a novel deep learning-based style transfer method and present target feature palette and attention coloring modules, and provide in-depth analysis and insight on the proposed method via exhaustive ablation study. Qualitative and quantitative results show that our stylized images exhibit state-of-the-art performance, with strength in preserving core structures and details of the content image.
ISSN:1380-7501
1573-7721
DOI:10.1007/s11042-022-13123-4