Deep learning based basketball video analysis for intelligent arena application

• Published in: Multimedia tools and applications 2017-12, Vol.76 (23), p.24983-25001
• Main Authors: Liu, Wu, Yan, Chenggang Clarence, Liu, Jiangyu, Ma, Huadong
• Format: Article
• Language: English
• Subjects: Basketball; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Deep learning; Downloading; Electronic devices; Image detection; Multimedia; Multimedia Information Systems; Neural networks; Reels; Special Purpose and Application-Based Systems; Stadiums; Video broadcasting
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-017-5002-5

Given the tremendous growth of sport fans, the “Intelligent Arena”, which can greatly improve the fun of traditional sports, becomes one of the new-emerging applications and research topics. The development of multimedia computing and artificial intelligence technologies support intelligent sport video analysis to add live video broadcast, score detection, highlight video generation, and online sharing functions to the intelligent arena applications. In this paper, we have proposed a deep learning based video analysis scheme for intelligent basketball arena applications. First of all, with multiple cameras or mobile devices capturing the activities in arena, the proposed scheme can automatically select the camera to give high-quality broadcast in real-time. Furthermore, with basketball energy image based deep conventional neural network, we can detect the scoring clips as the highlight video reels to support the wonderful actions replay and online sharing functions. Finally, evaluations on a built real-world basketball match dataset demonstrate that the proposed system can obtain 94.59 % accuracy with only less than 45 m s processing time (i.e., 10 m s broadcast camera selection, and 35 m s for scoring detection) for each frame. As the outstanding performance, the proposed deep learning based basketball video analysis scheme is implemented into a commercial intelligent basketball arena application named “Standz Basketball”. Although the application had been only released for one month, it achieves the 85 t h day download ranking place in the sport category of Chinese iTunes market.