CAT-DM: Controllable Accelerated Virtual Try-On with Diffusion Model

• Main Authors: Zeng, Jianhao, Song, Dan, Nie, Weizhi, Tian, Hongshuo, Wang, Tongtong, Liu, An-An
• Format: Conference Proceeding
• Language: English
• Subjects: Controllability; Diffusion models; Generative adversarial networks; Image quality; Image resolution; Noise reduction
• ISSN: 2575-7075
• DOI: 10.1109/CVPR52733.2024.00800

Generative Adversarial Networks (GANs) dominate the research field in image-based virtual try-on, but have not resolved problems such as unnatural deformation of garments and the blurry generation quality. While the generative quality of diffusion models is impressive, achieving controllability poses a significant challenge when applying it to virtual try-on and multiple denoising iterations limit its potential for real-time applications. In this paper, we propose Controllable Accelerated virtual Try-on with Diffusion Model (CAT-DM). To enhance the controllability, a basic diffusion-based virtual try-on network is designed, which utilizes ControlNet to introduce additional control conditions and improves the feature extraction of garment images. In terms of acceleration, CAT-DM initiates a reverse denoising process with an implicit distribution generated by a pre-trained GAN-based model. Compared with previous try-on methods based on diffusion models, CAT-DM not only retains the pattern and texture details of the in-shop garment but also reduces the sampling steps without compromising generation quality. Extensive experiments demonstrate the superiority of CAT-DM against both GAN-based and diffusion-based methods in producing more real-istic images and accurately reproducing garment patterns.