HEVC-EPIC: Fast Optical Flow Estimation From Coded Video via Edge-Preserving Interpolation

This paper presents a method leveraging coded motion information to obtain fast, high quality motion field estimation. The method is inspired by a recent trend followed by a number of top-performing optical flow estimation schemes that first estimate a sparse set of features between two frames, and...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on image processing 2018-06, Vol.27 (6), p.3100-3113
Main Authors: Rufenacht, Dominic, Taubman, David
Format: Article
Language:English
Subjects:
Adaptive optics
coded motion
edge-aware interpolation
Estimation
HEVC
Interpolation
low-complexity
Motion estimation
Optical distortion
optical flow
Optical imaging
Video compression
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper presents a method leveraging coded motion information to obtain fast, high quality motion field estimation. The method is inspired by a recent trend followed by a number of top-performing optical flow estimation schemes that first estimate a sparse set of features between two frames, and then use an edge-preserving interpolation scheme (EPIC) to obtain a piecewise-smooth motion field that respects moving object boundaries. In order to skip the time-consuming estimation of features, we propose to directly derive motion seeds from decoded HEVC block motion; we call the resulting scheme "HEVC-EPIC". We propose motion seed weighting strategies that account for the fact that some motion seeds are less reliable than others. Experiments on a large variety of challenging sequences and various bit-rates show that HEVC-EPIC runs significantly faster than EPIC flow, while producing motion fields that have a slightly lower average endpoint error. HEVC-EPIC opens the door of seamlessly integrating HEVC motion into video analysis and enhancement tasks. When employed as input to a framerate upsampling scheme, the average Y-PSNR of the interpolated frames using HEVC-EPIC motion slightly outperforms EPIC flow across the tested bit-rates, while running an order of magnitude faster.
ISSN:1057-7149
1941-0042
DOI:10.1109/TIP.2018.2813090