AI Naturalists Might Hold the Key to Unlocking Biodiversity Data in Social Media Imagery

• Published in: Patterns (New York, N.Y.) N.Y.), 2020-10, Vol.1 (7), p.100116-100116, Article 100116
• Main Authors: August, Tom A., Pescott, Oliver L., Joly, Alexis, Bonnet, Pierre
• Format: Article
• Language: English
• Subjects: Artificial Intelligence; big data; biodiversity; Biodiversity and Ecology; botany; Computer Science; computer vision; Computer Vision and Pattern Recognition; deep learning; Environment and Society; Environmental Sciences; informatics; Machine Learning; Modeling and Simulation; Multimedia; plants; social media
• ISSN: 2666-3899; 2666-3899
• DOI: 10.1016/j.patter.2020.100116

The increasing availability of digital images, coupled with sophisticated artificial intelligence (AI) techniques for image classification, presents an exciting opportunity for biodiversity researchers to create new datasets of species observations. We investigated whether an AI plant species classifier could extract previously unexploited biodiversity data from social media photos (Flickr). We found over 60,000 geolocated images tagged with the keyword “flower” across an urban and rural location in the UK and classified these using AI, reviewing these identifications and assessing the representativeness of images. Images were predominantly biodiversity focused, showing single species. Non-native garden plants dominated, particularly in the urban setting. The AI classifier performed best when photos were focused on single native species in wild situations but also performed well at higher taxonomic levels (genus and family), even when images substantially deviated from this. We present a checklist of questions that should be considered when undertaking a similar analysis. •AI image classifiers can create biodiversity datasets from social media imagery•Flickr hosts many images of plants; some can be accurately classified to species by AI•Images are spatially aggregated around tourist sites and under-represent native species•Images focused on a single, non-horticultural, plant are most reliably identified Recent reports of global biodiversity decline make it more important than ever to monitor biodiversity so that we can detect changes and infer their drivers. Online digital media, such as social media images, may be a new source of biodiversity observations, but they are far too numerous for a human to practically review. In this paper we apply an AI image classifier, designed to identify plants from images, to social media imagery to assess this method as a way to generate new biodiversity observations. We find that this approach is able to generate new data on species occurrence but that there are biases in both the social media data and the AI image classifier that need to be considered in analyses. This approach could be applied outside the biodiversity domain, to any phenomena of interest that may be captured in social media imagery. The checklist we provide at the end of this paper should therefore be of interest to anyone considering this approach to generating new data. We apply newly developed AI image classifiers to large social media image datase