Image tag completion via dual-view linear sparse reconstructions

• Published in: Computer vision and image understanding 2014-07, Vol.124, p.42-60
• Main Authors: Lin, Zijia, Ding, Guiguang, Hu, Mingqing, Lin, Yunzhen, Sam Ge, Shuzhi
• Format: Article
• Language: English
• Subjects: Correlation; DLSR; Image reconstruction; Image tag completion; Image tagging; Linear sparse reconstruction; Marking; Mathematical analysis; Reconstruction; Tag refinement; Tags; Vectors (mathematics)
• ISSN: 1077-3142; 1090-235X
• DOI: 10.1016/j.cviu.2014.03.012

•We propose a tag completion method via dual-view linear sparse reconstructions.•The proposed method can be used as an inductive method or a transductive one.•Various available contextual information should be considered for tag completion.•The proposed method yields the state-of-the-art performance for tag completion. User-provided textual tags of web images are widely utilized for facilitating image management and retrieval. Yet they are usually incomplete and insufficient to describe the whole semantic content of the corresponding images, resulting in performance degradations of various tag-dependent applications. In this paper, we propose a novel method denoted as DLSR for automatic image tag completion via Dual-view Linear Sparse Reconstructions. Given an incomplete initial tagging matrix with each row representing an image and each column representing a tag, DLSR performs tag completion from both views of image and tag, exploiting various available contextual information. Specifically, for a to-be-completed image, DLSR exploits image-image correlations by linearly reconstructing its low-level image features and initial tagging vector with those of others, and then utilizes them to obtain an image-view reconstructed tagging vector. Meanwhile, by linearly reconstructing the tagging column vector of each tag with those of others, DLSR exploits tag-tag correlations to get a tag-view reconstructed tagging vector with the initially labeled tags. Then both image-view and tag-view reconstructed tagging vectors are combined for better predicting missing related tags. Extensive experiments conducted on benchmark datasets and real-world web images well demonstrate the reasonableness and effectiveness of the proposed DLSR. And it can be utilized to enhance a variety of tag-dependent applications such as image auto-annotation.