Sandwiched Image Compression: Wrapping Neural Networks Around A Standard Codec

We sandwich a standard image codec between two neural networks: a preprocessor that outputs neural codes, and a postprocessor that reconstructs the image. The neural codes are compressed as ordinary images by the standard codec. Using differentiable proxies for both rate and distortion, we develop a...

Full description

Saved in:
Bibliographic Details
Main Authors: Guleryuz, Onur G., Chou, Philip A., Hoppe, Hugues, Tang, Danhang, Du, Ruofei, Davidson, Philip, Fanello, Sean
Format: Conference Proceeding
Language:English
Subjects:
Codecs
Computer architecture
deep learning
Image coding
image compression
Neural networks
Rate distortion theory
Rate-distortion
Wrapping
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We sandwich a standard image codec between two neural networks: a preprocessor that outputs neural codes, and a postprocessor that reconstructs the image. The neural codes are compressed as ordinary images by the standard codec. Using differentiable proxies for both rate and distortion, we develop a rate-distortion optimization framework that trains the networks to generate neural codes that are efficiently compressible as images. This architecture not only improves rate-distortion performance for ordinary RGB images, but also enables efficient compression of alternative image types (such as normal maps of computer graphics) using standard image codecs. Results demonstrate the effectiveness and flexibility of neural processing in mapping a variety of input data modalities to the rigid structure of standard codecs. A surprising result is that the rate-distortion-optimized neural processing seamlessly learns to transport color images using a single-channel (grayscale) codec.
ISSN:2381-8549
DOI:10.1109/ICIP42928.2021.9506256