A proxy-data-based hierarchical adversarial patch generation method

• Published in: Computer vision and image understanding 2024-09, Vol.246, p.104066, Article 104066
• Main Authors: Liu, Jiawei, Gong, Xun, Wang, Tingting, Hu, Yunfeng, Chen, Hong
• Format: Article
• Language: English
• Subjects: Adversarial patch; Data privacy; Physical adversarial attack; Proxy dataset
• ISSN: 1077-3142; 1090-235X
• DOI: 10.1016/j.cviu.2024.104066

Current training data-dependent physical attacks have limited applicability to privacy-critical situations when attackers lack access to neural networks’ training data. To address this issue, this paper presents a hierarchical adversarial patch generation framework considering data privacy, utilizing proxy datasets while assuming that the training data is blinded. In the upper layer, Average Patch Saliency (APS) is introduced as a quantitative metric to determine the best proxy dataset for patch generation from a set of publicly available datasets. In the lower layer, Expectation of Transformation Plus (EoT+) method is developed to generate patches while accounting for perturbing background simulation and sensitivity alleviation. Evaluation results obtained in digital settings show that the proposed proxy-data-based framework achieves comparable targeted attack results to the data-dependent benchmark method. Finally, the framework’s validity is comprehensively evaluated in the physical world, where the corresponding experimental videos and code can be found at here. •A proxy-data-based physical attack framework to reconcile robustness evaluation and private data protection.•A novel metric for determining the proxy dataset form publicly available datasets.•A new adversarial patch generation method that allows for lower limit improvement on adversarial strength.