Online Data Organizer: Micro-Video Categorization by Structure-Guided Multimodal Dictionary Learning

• Published in: IEEE transactions on image processing 2019-03, Vol.28 (3), p.1235-1247
• Main Authors: Liu, Meng, Nie, Liqiang, Wang, Xiang, Tian, Qi, Chen, Baoquan
• Format: Article
• Language: English
• Subjects: Acoustics; Computer science; Dictionaries; Digital media; Distance learning; Machine learning; Micro-video organization; multi-modal dictionary learning; online learning; Social network services; Social networks; Taxonomy; tree-guided constraints; Videos; Visualization
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2018.2875363

Micro-videos have rapidly become one of the most dominant trends in the era of social media. Accordingly, how to organize them draws our attention. Distinct from the traditional long videos that would have multi-site scenes and tolerate the hysteresis, a micro-video: (1) usually records contents at one specific venue within a few seconds. The venues are structured hierarchically regarding their category granularity. This motivates us to organize the micro-videos via their venue structure. (2) timely circulates over social networks. Thus, the timeliness of micro-videos desires effective online processing. However, only 1.22% of micro-videos are labeled with venue information when uploaded at the mobile end. To address this problem, we present a framework to organize the micro-videos online. In particular, we first build a structure-guided multi-modal dictionary learning model to learn the concept-level micro-video representation by jointly considering their venue structure and modality relatedness. We then develop an online learning algorithm to incrementally and efficiently strengthen our model, as well as categorize the micro-videos into a tree structure. Extensive experiments on a real-world data set validate our model well. In addition, we have released the codes to facilitate the research in the community.