AI-Based Cropping of Ice Hockey Videos for Different Social Media Representations

• Published in: IEEE access 2024
• Main Authors: Houshmand Sarkhoosh, Mehdi, Dorcheh, Sayed Mohammad Majidi, Midoglu, Cise, Sabet, Saeed Shafiei, Kupka, Tomas, Johansen, Dag, Riegler, Michael Alexander, Halvorsen, Pål
• Format: Article
• Language: English
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2024.3449152

Sports multimedia is among the most prominent types of content distributed across social media today, and the retargeting of videos for diverse aspect ratios is essential for a suitable representation on different social media platforms. In this respect, ice hockey is quite challenging due to its agile movement pattern and speed, and because the main reference point (puck) is very small. In this paper, we introduce a novel pipeline for intelligent video cropping tailored for ice hockey. Our main goal is to identify regions of interest in video frames by detecting and tracking the hockey puck using state-of-the-art AI models. Our pipeline employs scene detection, object detection, outlier detection, and smoothing as key features. Our proposed pipeline called SmartCrop-H is not only highly efficient and configurable with respect to target aspect ratios, but also addresses the automation needs in this domain. Our comprehensive evaluation, comprising objective and subjective measures, shows the overall efficiency of the entire pipeline, including assessments of both the individual components and the end-to-end system performance. We also demonstrate the practical applicability of SmartCrop-H with a user study, which indicates that our framework performs on par with, or even surpasses, professional tools in terms of output quality.