ClickSmart: A Context-Aware Viewpoint Recommendation System for Mobile Photography

• Published in: IEEE transactions on circuits and systems for video technology 2017-01, Vol.27 (1), p.149-158
• Main Authors: Rawat, Yogesh Singh, Kankanhalli, Mohan S.
• Format: Article
• Language: English
• Subjects: Cameras; Context; generative model; Media; Meteorology; mobile photography; Photography; Real-time systems; Recommender systems; sensors; social media; viewpoint; Visualization
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2016.2555658

In this paper, we propose ClickSmart, a viewpoint recommendation system that can assist a user in capturing high-quality photographs at well-known tourist locations. ClickSmart can provide real-time viewpoint recommendation based on the preview on the user's camera, current time, and user's geolocation. It makes use of publicly available geotagged images along with the associated metadata for learning a recommendation model. We define view-cells, macroblocks in geospace, and propose the concepts of popularity, quality, and uniqueness of view-cells from the viewpoint perspective. Viewpoint recommendation is generated at the granularity of a view-cell and is based on its popularity, quality, and uniqueness, which are estimated using social media cues associated with images. We further observe that contextual information such as time and weather conditions play an important role in photography, and therefore augment the recommendation system with the associated context. ClickSmart also takes into account the presence of people in the view for making the recommendation. It can provide two kinds of recommendations, quality based and uniqueness based. Although both were found effective in the experimental evaluation, our user study showed that uniqueness-based recommendation was preferred more by skilled photographers compared with amateurs.