Standard Compatible Efficient Video Coding with Jointly Optimized Neural Wrappers

We present a standard-compatible video coding scheme with end-to-end optimized neural wrapper over standard video codecs that achieves significant rate-distortion (R-D) performance gains and is still efficient in decoding. We train a pair of pre- and post-processor using a differential JPEG proxy. T...

Full description

Saved in:
Bibliographic Details
Main Authors: Hu, Yueyu, Zhang, Chenhao, Guleryuz, Onur G., Mukherjee, Debargha, Wang, Yao
Format: Conference Proceeding
Language:English
Subjects:
Convolution
Efficient Video Coding
Laplace equations
Neural Network
Performance gain
Postprocess
Preprocess
Rate-distortion
Transform coding
Transforms
Video coding
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a standard-compatible video coding scheme with end-to-end optimized neural wrapper over standard video codecs that achieves significant rate-distortion (R-D) performance gains and is still efficient in decoding. We train a pair of pre- and post-processor using a differential JPEG proxy. The pre-processor applies a learned transform to the video and downsamples the video by a factor of 2. It generates a bottleneck video to be coded by a standard codec as a YUV sequence. The post-processor takes the decoded bottleneck video, does the inverse transform, and upsamples it to the original resolution. We follow the design in [1] , where we configure downsample using a layer of strided convolution. We optimize the post-processor for efficiency by replacing convolutions with kernel size larger than 1×1 to depth-wise convolutions [2] .
ISSN:2375-0359
DOI:10.1109/DCC58796.2024.00078