Towards Analysis-Friendly Face Representation With Scalable Feature and Texture Compression

Compactly representing visual information plays a fundamental role in optimizing the ultimate utility of myriad visual data-centered applications. Numerous approaches have been proposed to efficiently compress the texture and visual features for human visual perception and machine intelligence, resp...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on multimedia 2022, Vol.24, p.3169-3181
Main Authors: Wang, Shurun, Wang, Shiqi, Yang, Wenhan, Zhang, Xinfeng, Wang, Shanshe, Ma, Siwei, Gao, Wen
Format: Article
Language:English
Subjects:
Artificial neural networks
Deep learning
Feature compression
Feature extraction
Image coding
Image compression
Image reconstruction
Machine learning
multitask learning
Neural networks
Optimization
Representations
scalable coding
Texture
texture compression
Transform coding
Visual perception
Visualization
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Compactly representing visual information plays a fundamental role in optimizing the ultimate utility of myriad visual data-centered applications. Numerous approaches have been proposed to efficiently compress the texture and visual features for human visual perception and machine intelligence, respectively; however, much less work has been dedicated to studying the interactions between them. Here, we investigate the integration of feature and texture compression and show that a universal and collaborative visual information representation can be achieved in a hierarchical way. In particular, we study feature and texture compression in a scalable coding framework, where the base layer serves as the deep learning feature and the enhancement layer targets to perfectly reconstruct the texture. Based on the strong generative capability of deep neural networks, the gap between the base feature layer and enhancement layer is further filled with feature-level texture reconstruction, with the goal of further constructing texture representations from features. As such, the residuals between the original and reconstructed texture could be further conveyed in the enhancement layer. To improve the efficiency of the proposed framework, the base layer neural network is trained in a multitask manner such that the learned features enjoy both high-quality reconstruction and high-accuracy analysis. The framework and optimization strategies are further applied in face image compression, and promising coding performance has been achieved in terms of both rate-fidelity and rate-accuracy evaluations.
ISSN:1520-9210
1941-0077
DOI:10.1109/TMM.2021.3094300