A Lightweight Model for Deep Frame Prediction in Video Coding

Recent studies have demonstrated the efficacy of deep neural network (DNN)-based inter frame prediction for video coding. The network commonly used in these studies is built upon a U-Net-like architecture and produces content-adaptive 1-D separable filters with a large number of taps for frame predi...

Full description

Saved in:
Bibliographic Details
Main Authors: Choi, Hyomin, Bajic, Ivan V.
Format: Conference Proceeding
Language:English
Subjects:
Convolution
Convolutional codes
deep frame prediction
deep neural network
Neural networks
Pipelines
Rate-distortion
Technological innovation
Video coding
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Recent studies have demonstrated the efficacy of deep neural network (DNN)-based inter frame prediction for video coding. The network commonly used in these studies is built upon a U-Net-like architecture and produces content-adaptive 1-D separable filters with a large number of taps for frame prediction. This leads to a model with a large number of parameters. In this paper, we propose a lighter version of the network with significantly fewer parameters, by making use of dilated convolutional layers and making the U-Net shallower. In addition, we introduce a DCT-based ℓ 1 -loss term that encourages compression, and explore several ways of integrating our lightweight model into HEVC. Both frame prediction accuracy and coding efficiency are compared against previous works. The experiments show that the proposed model achieves up to 6.4% average bit reduction in terms of BD-Bitrate against HEVC, which is significantly better than existing methods in the literature.
ISSN:2576-2303
DOI:10.1109/IEEECONF51394.2020.9443427