Single-stream long-term optical flow convolution network for action recognition of lameness dairy cow

• Published in: Computers and electronics in agriculture 2020-08, Vol.175, p.105536, Article 105536
• Main Authors: Jiang, Bo, Yin, Xuqiang, Song, Huaibo
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial neural networks; Bovine spongiform encephalopathy; Convolution; Dairy cow; Fields (mathematics); Lameness recognition; Long-term; Neural networks; Optical communication; Optical flow; Optical flow (image analysis); Recognition; Representations; Video
• ISSN: 0168-1699; 1872-7107
• DOI: 10.1016/j.compag.2020.105536

•A deep learning-based method was proposed for the recognition cow lameness.•Single-stream optical flow convolution model can recognize cow lameness effectively.•The lameness behavior recognition accuracy of the proposed method was 98.24%.•The computational efficiency can achieve 564FPS and 1.77 ms/image.•This method was more stable in the antagonism of environmental robustness. Typical dairy cow lameness actions are only several seconds in duration and exhibit characteristic spatiotemporal structures. Manual lameness detection poses several problems, such as difficulty in the detection of sudden, severe or early lameness behavior. In this paper, our proposed method attempts to capture this structure and to learn action representations with convolutional neural networks. However, such representations are typically learned at the level of a few video frames, thereby failing to model actions over their full temporal extent. In this study, we learn video representations using neural networks with single-stream long-term optical flow convolution. To evaluate the performance of the algorithm, 756 videos were randomly selected from 1080 dairy cow videos as the original training videos, and the remaining videos were used as test videos. The experimental results demonstrate that single-stream long-term optical flow convolution network models with increased temporal extents improve the accuracy of dairy cow lameness action recognition. We also study the impacts of various low-level representations, such as raw values of video pixels and optical flow vector fields, and we demonstrate the importance of high-quality optical flow estimation for learning accurate dairy cow lameness action models. We report state-of-the-art results on challenging benchmarks for the dairy cow lameness action video set (98.24%).