Fashion intelligence system: An outfit interpretation utilizing images and rich abstract tags

• Published in: Expert systems with applications 2023-03, Vol.213, p.119167, Article 119167
• Main Authors: Shimizu, Ryotaro, Saito, Yuki, Matsutani, Megumi, Goto, Masayuki
• Format: Article
• Language: English
• Subjects: Attribute activation map; Fashion intelligence; Fashion interpretation; Image retrieval; Outfit image; Visual-semantic embedding
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2022.119167

In recent years, it has become common for consumers to familiarize themselves with the latest fashion trends through the internet and engage in their own fashion-inspired shopping activities. Therefore, making fashion-inspired shopping and browsing activities (internet surfing in the fashion domain) comfortable is essential because it leads to interactions in the fashion industry. However, fashion is a fuzzy and complex domain that contains many abstract elements, and this ambiguity and complexity can hinder users’ deep interest in the fashion industry. Therefore, we define a novel technology and domain called “fashion intelligence” and propose a system based on a visual-semantic embedding method for automatically learning and interpreting fashion and obtaining answers to users’ questions. Our proposed method can embed the abundant abstract tag information in the same projective space as outfit images. Mapping of images and tags in a projective space helps search for outfit images using fashion-specific abstract words. In addition, visually estimating the degree of relevance between images and tags helps interpret abstract words. As a result, this research helps decrease fashion-specific ambiguity and complexity and supports the marketing activities and fashion choices of both experts and non-experts. •A novel technology and research area called “fashion intelligence” is defined.•Full-body outfit images are embedded with rich fashion-specific abstract tags.•Effective learning methods with broad, complex backgrounds are presented.•Various applications of the system for real-world service are introduced.•Interpretation of fashion using the proposed system is demonstrated.