Inheritance Attention Matrix-Based Universal Adversarial Perturbations on Vision Transformers

Universal Adversarial Perturbations (UAPs), which is type of image-agnostic adversarial attack, has been deeply investigated for Convolutional Neural Networks due to its high efficiency. On the other hand, as an architecture based on self-attention mechanism, Vision Transformers (ViTs) have boomed a...

Full description

Saved in:
Bibliographic Details
Published in:IEEE signal processing letters 2021, Vol.28, p.1923-1927
Main Authors: Hu, Haoqi, Lu, Xiaofeng, Zhang, Xinpeng, Zhang, Tianxing, Sun, Guangling
Format: Article
Language:English
Subjects:
Artificial neural networks
Computer vision
deep learning
Head
Inheritances
Optimization
Perturbation
Perturbation methods
Robustness
self-attention
Task analysis
Transformers
universal adversarial perturbations
Vision Transformers
Visualization
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Universal Adversarial Perturbations (UAPs), which is type of image-agnostic adversarial attack, has been deeply investigated for Convolutional Neural Networks due to its high efficiency. On the other hand, as an architecture based on self-attention mechanism, Vision Transformers (ViTs) have boomed and been widely applied to solve various computer vision problems since most recent years. In this letter, we delve into the robustness of ViTs against universal adversarial attack and propose an inheritance attention matrix-based UAPs (IAM-UAP). Specifically, we introduce the inheritance attention weight matrix (IAM), which represents the integration of global information of the input patch sequence. Further, we propose a perturbation optimization objective based on IAM to confuse the global information integration of ViTs. The empirical results confirm the attacking capability of IAM-UAP on ViTs with a moderate attacking rate. In addition, we also disclose that the patch size of ViTs is a latent factor influencing the robustness against the universal attack.
ISSN:1070-9908
1558-2361
DOI:10.1109/LSP.2021.3112099