Unveiling Real-Life Effects of Online Photo Sharing

• Main Authors: Nguyen, Van-Khoa, Popescu, Adrian, Deshayes-Chossart, Jerome
• Format: Conference Proceeding
• Language: English
• Subjects: Computer vision; Correlation; Crowdsourcing; Explainable AI; Fairness; Accountability; Privacy and Ethics in Vision Datasets; Evaluation and Comparison of Vision Algorithms; Manuals; Object detection; Social networking (online); Visualization
• ISSN: 2642-9381
• DOI: 10.1109/WACV51458.2022.00391

Social networks give free access to their services in exchange for the right to exploit their users' data. Data sharing is done in an initial context which is chosen by the users. However, data are used by social networks and third parties in different contexts which are often not transparent. In order to unveil such usages, we propose an approach which focuses on the effects of data sharing in impactful real-life situations. Focus is put on visual content because of its strong influence in shaping online user profiles. The approach relies on three components: (1) a set of visual objects with associated situation impact ratings obtained by crowdsourcing, (2) a corresponding set of object detectors for mining users' photos and (3) a ground truth dataset made of 500 visual user profiles which are manually rated per situation. These components are combined in LERV UP, a method which learns to rate visual user profiles in each situation. LERV UP exploits a new image descriptor which aggregates object ratings and object detections at user level and an attention mechanism which boosts highly-rated objects to prevent them from being overwhelmed by low-rated ones. Performance is evaluated per situation by measuring the correlation between the automatic ranking of profile ratings and a manual ground truth. Results indicate that LERV UP is effective since a strong correlation of the two rankings is obtained. A practical implementation of the approach in a mobile app which raises user awareness about shared data usage is also discussed.