Generating and Controlling Diversity in Image Search

• Main Authors: Mehrab Tanjim, Md, Sinha, Ritwik, Singh, Krishna Kumar, Mahadevan, Sridhar, Arbour, David, Sinha, Moumita, Cottrell, Garrison W.
• Format: Conference Proceeding
• Language: English
• Subjects: Computational modeling; Computer vision; Explainable AI; Fairness; Accountability; Privacy and Ethics in Vision Datasets; Evaluation and Comparison of Vision Algorithms; Deep Learning -> Neural Generative Models; Autoencoders; GANs; Large-scale Vision Applications; Image resolution; Image synthesis; Measurement; Process control; Training
• ISSN: 2642-9381
• DOI: 10.1109/WACV51458.2022.00396

In our society, generations of systemic biases have led to some professions being more common among certain genders and races. This bias is also reflected in image search on stock image repositories and search engines, e.g., a query like "male Asian administrative assistant" may produce limited results. The pursuit of a utopian world demands providing content users with an opportunity to present any profession with diverse racial and gender characteristics. The limited choice of existing content for certain combinations of profession, race, and gender presents a challenge to content providers. Current research dealing with bias in search mostly focuses on re-ranking algorithms. However, these methods cannot create new content or change the overall distribution of protected attributes in photos. To remedy these problems, we propose a new task of high-fidelity image generation conditioning on multiple attributes from imbalanced datasets. Our proposed task poses new sets of challenges for the state-of-the-art Generative Adversarial Networks (GANs). In this paper, we also propose a new training framework to better address the challenges. We evaluate our framework rigorously on a real-world dataset and perform user studies that show our model is preferable to the alternatives.