Gradient Difference-Predictive Coding Network (GD-PredNet): Reducing the Convergence Time for Future Frame Prediction with Infinitesimal Quality Trade-off

• Main Authors: Mikkilineni, Sai Ranganath, Privat, Taylor D., Totaro, Michael Wayne
• Format: Conference Proceeding
• Language: English
• Subjects: Brain modeling; Crowd Sourced Dataset; Future Frame Prediction; Gradient Difference; Loss Functions; Measurement uncertainty; Predictive coding; Predictive models; PredNet; Something-Something dataset; Training; Transforms; Unsupervised Learning; Videos; Visualization
• ISSN: 1558-058X
• DOI: 10.1109/SoutheastCon48659.2022.9763950

Even though future frame prediction in videos is a relatively young unsupervised learning task, it has shown promise by accommodating the networks to effectively learn efficient internal representations in a visual hyperspace. Predictive Coding Network (PredNet) uses future frame predictions as a learning signal and has a legacy background of unconscious inference, free energy, and predictive coding model of the visual cortex; it is still a relatively young network compared to RNNs, CNNs, and so on. Although Rao and Ballard's proposed Predictive Coding (PC) model is aimed at reducing the redundancy within the learned internal representations by a network, and Lotter et al.'s design of the PredNet might not be the ideal replication of the PC model, it still shows promise for learning better less-redundant internal representations than other networks. In this paper, we augment PredNet to enhance its performance in future frame prediction. Additionally, we introduce a new measure known as the gradient difference error (GDE) measure based on the gradient difference loss (GDL) function proposed by Mathieu et al. We do this to adapt the GDL function to the context of PredNet since it uses an implicit loss function besides the explicit loss used during training. Our experimental results show that PredNet, when using a combination of the L1 loss function with GDE or GDL, is faster to converge to the best performance while trading off minimal quality of the predictions within a given training window. In doing so, we transform PredNet into Gradient Difference-PredNet (GD-PredNet), and we aim to encourage increased research in Predictive Coding and PredNet.