Robustness of Generative Adversarial CLIPs Against Single-Character Adversarial Attacks in Text-to-Image Generation

Generative Adversarial Networks (GANs) have emerged as a powerful type of generative model, particularly effective at creating images from textual descriptions. Similar to diffusion models, GANs rely on text encoders to derive embeddings from these descriptions. However, this reliance introduces spe...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2024-01, Vol.12, p.1-1
Main Authors: Chanakya, Patibandla, Harsha, Putla, Singh, Krishna Pratap
Format: Article
Language:English
Subjects:
CLIP text encoder
Coders
Degradation
Descriptions
Diffusion models
GALIP
GAN
Generative adversarial networks
Generators
Image degradation
Image processing
Image quality
Image synthesis
Performance degradation
Performance evaluation
Perturbation methods
Robustness
Single-character attack
Text to image
Text-to-image generation
Training
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Generative Adversarial Networks (GANs) have emerged as a powerful type of generative model, particularly effective at creating images from textual descriptions. Similar to diffusion models, GANs rely on text encoders to derive embeddings from these descriptions. However, this reliance introduces specific vulnerabilities to adversarial attacks. A notable example is a single-character adversarial attack, where altering a single character in the text description can lead to significant performance degradation in the generated image quality and model's performance. In this study, we systematically evaluate the susceptibility of GANs to such attacks using GALIP, a single-stage architecture that leverages a pre-trained CLIP text encoder for text embeddings. We meticulously selected captions with single-character modifications that exhibit maximum and median-distance embeddings for the attack. Experimental results show up to 310.5% degradation in Frechet Inception Distance (FID) scores, underscoring the importance of developing improved defenses in text-to-image synthesis.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2024.3491017