AIpparel: A Large Multimodal Generative Model for Digital Garments

Apparel is essential to human life, offering protection, mirroring cultural identities, and showcasing personal style. Yet, the creation of garments remains a time-consuming process, largely due to the manual work involved in designing them. To simplify this process, we introduce AIpparel, a large m...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-12
Main Authors: Nakayama, Kiyohiro, Ackermann, Jan, Timur Levent Kesdogan, Yang, Zheng, Korosteleva, Maria, Sorkine-Hornung, Olga, Guibas, Leonidas J, Yang, Guandao, Wetzstein, Gordon
Format: Article
Language:English
Subjects:
Annotations
Editing
Garments
Sewing
Task complexity
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Apparel is essential to human life, offering protection, mirroring cultural identities, and showcasing personal style. Yet, the creation of garments remains a time-consuming process, largely due to the manual work involved in designing them. To simplify this process, we introduce AIpparel, a large multimodal model for generating and editing sewing patterns. Our model fine-tunes state-of-the-art large multimodal models (LMMs) on a custom-curated large-scale dataset of over 120,000 unique garments, each with multimodal annotations including text, images, and sewing patterns. Additionally, we propose a novel tokenization scheme that concisely encodes these complex sewing patterns so that LLMs can learn to predict them efficiently. AIpparel achieves state-of-the-art performance in single-modal tasks, including text-to-garment and image-to-garment prediction, and enables novel multimodal garment generation applications such as interactive garment editing. The project website is at georgenakayama.github.io/AIpparel/.
ISSN:2331-8422