Adversarial Examples Generation for Deep Product Quantization Networks on Image Retrieval

• Published in: IEEE transactions on pattern analysis and machine intelligence 2023-02, Vol.45 (2), p.1388-1404
• Main Authors: Chen, Bin, Feng, Yan, Dai, Tao, Bai, Jiawang, Jiang, Yong, Xia, Shu-Tao, Wang, Xuan
• Format: Article
• Language: English
• Subjects: adversarial attack; Artificial neural networks; Centroids; deep learning; Feature extraction; Image classification; Image retrieval; KL divergence; Measurement; Perturbation; Perturbation methods; Probability distribution; Product quantization; Quantization (signal); Search engines; Source code; Task analysis
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2022.3165024

Deep product quantization networks (DPQNs) have been successfully used in image retrieval tasks, due to their powerful feature extraction ability and high efficiency of encoding high-dimensional visual features. Recent studies show that deep neural networks (DNNs) are vulnerable to input with small and maliciously designed perturbations (a.k.a., adversarial examples) for classification. However, little effort has been devoted to investigating how adversarial examples affect DPQNs, which raises the potential safety hazard when deploying DPQNs in a commercial search engine. To this end, we propose an adversarial example generation framework by generating adversarial query images for DPQN-based retrieval systems. Unlike the adversarial generation for the classic image classification task that heavily relies on ground-truth labels, we alternatively perturb the probability distribution of centroids assignments for a clean query, then we can induce effective non-targeted attacks on DPQNs in white-box and black-box settings. Moreover, we further extend the non-targeted attack to a targeted attack by a novel sample space averaging scheme (\text{S}^{2} S2 AS), whose theoretical guarantee is also obtained. Extensive experiments show that our methods can create adversarial examples to successfully mislead the target DPQNs. Besides, we found that our methods both significantly degrade the retrieval performance under a wide variety of experimental settings. The source code is available at https://github.com/Kira0096/PQAG .