Identifying crisis-related informative tweets using learning on distributions

• Published in: Information processing & management 2020-03, Vol.57 (2), p.102145, Article 102145
• Main Authors: Ghafarian, Seyed Hossein, Yazdi, Hadi Sadoghi
• Format: Article
• Language: English
• Subjects: Crisis incidents tweets; Crisis management; Experiments; Hypotheses; Identification methods; Information processing; Learning; Learning on distributions; Linguistics; Mathematical models; Social media; Social networks; Words (language)
• ISSN: 0306-4573; 1873-5371
• DOI: 10.1016/j.ipm.2019.102145

•Social network such as Twitter are valuable tools for sharing and informing others about an ongoing crisis.•Identifying crisis-related informative tweets is an essential tool for authorities to respond quickly to crisis.•Distributional hypothesis states that meaning similarity and distributional similarity are correlated.•Based on distributional hypothesis, each crisis-related tweet can be considered as a “distribution”.•Using the recent development in machine learning, namely, learning on distributions, each object of learning can be considered as a distribution.•Learning on distributions achieves very good results in identifying informative tweets about a crisis incident. Social networks like Twitter are good means for people to express themselves and ask for help in times of crisis. However, to provide help, authorities need to identify informative posts on the network from the vast amount of non-informative ones to better know what is actually happening. Traditional methods for identifying informative posts put emphasis on the presence or absence of certain words which has limitations for classifying these posts. In contrast, in this paper, we propose to consider the (overall) distribution of words in the post. To do this, based on the distributional hypothesis in linguistics, we assume that each tweet is a distribution from which we have drawn a sample of words. Building on recent developments in learning methods, namely learning on distributions, we propose an approach which identifies informative tweets by using distributional assumption. Extensive experiments have been performed on Twitter data from more than 20 crisis incidents of nearly all types of incidents. These experiments show the superiority of the proposed approach in a number of real crisis incidents. This implies that better modelling of the content of a tweet based on recent advances in estimating distributions and using domain-specific knowledge for various types of crisis incidents such as floods or earthquakes, may help to achieve higher accuracy in the task.