Panelformer: Sewing Pattern Reconstruction from 2D Garment Images

• Main Authors: Chen, Cheng-Hsiu, Su, Jheng-Wei, Hu, Min-Chun, Yao, Chih-Yuan, Chu, Hung-Kuo
• Format: Conference Proceeding
• Language: English
• Subjects: 3D computer vision; Algorithms; Computational modeling; Data augmentation; Image recognition and understanding; Point cloud compression; Shape; Three-dimensional displays; Transformers
• ISSN: 2642-9381
• DOI: 10.1109/WACV57701.2024.00051

In this paper, we present a novel approach for reconstructing garment sewing patterns from 2D garment images. Our method addresses the challenge of handling occlusion in 2D images by leveraging the symmetric and correlated nature of garment panels. We introduce a transformer-based deep neural network called Panelformer that learns the parametric space of garment sewing patterns. The network comprises two components: the panel transformer and the stitch predictor. The panel transformer estimates the parametric panel shapes, including the occluded panels, by learning from the visible ones. The stitch predictor determines the stitching information among the predicted panels, enabling the reconstruction of the complete garment. To mitigate the overfitting problem caused by strong panel correlations, we propose two tailor-made data augmentation techniques: panel masking and garment mixing. These techniques generate a wider variety of panel combinations, enhancing the model's robustness and generalization capability. We evaluate the effectiveness of Panelformer using a synthetic dataset with diverse garment types. The experimental results demonstrate that our method outperforms competing baselines and achieves comparable performance to NeuralTailor, which operates on 3D point cloud data. This validates the efficacy of our approach in the context of garment sewing pattern reconstruction. By utilizing 2D images as input, our method expands the potential applications of garment modeling and offers easy accessibility to end users. Our code is available online 1 .