CHOP: An orthogonal hashing method for zero-shot cross-modal retrieval

• Published in: Pattern recognition letters 2021-05, Vol.145, p.247-253
• Main Authors: Yuan, Xu, Wang, Guangze, Chen, Zhikui, Zhong, Fangming
• Format: Article
• Language: English
• Subjects: Binary codes; Cross-modal retrieval; Knowledge management; Orthogonal projection; Retrieval; Zero-shot
• ISSN: 0167-8655; 1872-7344
• DOI: 10.1016/j.patrec.2021.02.016

•A new hashing method(CHOP) based on orthogonal projection is proposed.•Orthogonal-constrained and hash codes are used to connect seen and unseen classes.•CHOP can improve the discriminative of hash codes, and avoid the hubness problem.•Extensive experiments and results indicate the effectiveness of the proposed CHOP. Cross-modal retrieval has recently attracted much attention because it helps users retrieve data across different modalities. However, with the explosive growth of data, a large number of new emerging concepts (unseen classes) that have not been appeared in the training data (seen classes) bring great challenges to the traditional cross-modal retrieval. Nevertheless, most existing approaches mainly focus on improving cross-modal retrieval performance of seen classes, which may fail in the unseen classes. To address the challenge of zero-shot cross-modal retrieval, we propose an orthogonal method in this paper, i.e., Cross-modal Hashing with Orthogonal Projection (CHOP). It projects cross-modal features and class attributes onto a Hamming space, where each projection of cross-modal features is orthogonal to the mismatched class attributes. By so doing, the model can learn a discriminative and binary representation of each modality. In addition, the class attributes build a bridge to transfer knowledge from seen classes to unseen classes. Furthermore, the orthogonal constraint on binary codes can help to mitigate the hubness problem. Extensive experiments on three benchmark datasets show that the proposed CHOP is effective in handling zero-shot cross-modal retrieval.