Improving the classification of flood tweets with contextual hydrological information in a multimodal neural network

• Published in: Computers & geosciences 2020-07, Vol.140, p.104485, Article 104485
• Main Authors: de Bruijn, Jens A., de Moel, Hans, Weerts, Albrecht H., de Ruiter, Marleen C., Basar, Erkan, Eilander, Dirk, Aerts, Jeroen C.J.H.
• Format: Article
• Language: English
• Subjects: Authoritative data; Early warning; Floods; Multimodal fusion; Natural hazards; Neural networks; Text classification; Word embeddings
• ISSN: 0098-3004; 1873-7803
• DOI: 10.1016/j.cageo.2020.104485

While text classification can classify tweets, assessing whether a tweet is related to an ongoing flood event or not, based on its text, remains difficult. Inclusion of contextual hydrological information could improve the performance of such algorithms. Here, a multilingual multimodal neural network is designed that can effectively use both textual and hydrological information. The classification data was obtained from Twitter using flood-related keywords in English, French, Spanish and Indonesian. Subsequently, hydrological information was extracted from a global precipitation dataset based on the tweet's timestamp and locations mentioned in its text. Three experiments were performed analyzing precision, recall and F1-scores while comparing a neural network that uses hydrological information against a neural network that does not. Results showed that F1-scores improved significantly across all experiments. Most notably, when optimizing for precision the neural network with hydrological information could achieve a precision of 0.91 while the neural network without hydrological information failed to effectively optimize. Moreover, this study shows that including hydrological information can assist in the translation of the classification algorithm to unseen languages. •We combine hydrological and textual info to improve flood tweet classification.•The algorithm is multilingual and can be tailored to optimize precision or recall.•Hydrological information reduces the need for training data for a new language.•The algorithm can be used to improve flood detection and monitoring.