A survey of variational and CNN-based optical flow techniques

• Published in: Signal processing. Image communication 2019-03, Vol.72, p.9-24
• Main Authors: Tu, Zhigang, Xie, Wei, Zhang, Dejun, Poppe, Ronald, Veltkamp, Remco C., Li, Baoxin, Yuan, Junsong
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Challenges; CNN-based method; Computer terminals; Computer vision; Evaluation measures; Image processing; Optical flow; Optical flow (image analysis); Pedestrians; Variational method
• ISSN: 0923-5965; 1879-2677
• DOI: 10.1016/j.image.2018.12.002

Dense motion estimations obtained from optical flow techniques play a significant role in many image processing and computer vision tasks. Remarkable progress has been made in both theory and its application in practice. In this paper, we provide a systematic review of recent optical flow techniques with a focus on the variational method and approaches based on Convolutional Neural Networks (CNNs). These two categories have led to state-of-the-art performance. We discuss recent modifications and extensions of the original model, and highlight remaining challenges. For the first time, we provide an overview of recent CNN-based optical flow methods and discuss their potential and current limitations. •Introducing optical flow: the basic concepts, the characteristics of the variational and CNN-based techniques, and the evaluation measures.•Discussing developments of the variational method, analyzing the challenges and illustrating the corresponding treating strategies of it.•Describing the conception of the CNN-based technique, and give a detailed discussion of the issues of this technique.