Stoking the Flames: Understanding Escalation in an Online Harassment Community

• Published in: Proceedings of the ACM on human-computer interaction 2024-04, Vol.8 (CSCW1), p.1-23, Article 176
• Main Authors: Take, Kejsi, Zhong, Victoria, Geeng, Chris, Bevensee, Emmi, McCoy, Damon, Greenstadt, Rachel
• Format: Article
• Language: English
• Subjects: Human and societal aspects of security and privacy; Human computer interaction (HCI); Human-centered computing; Security and privacy; Social aspects of security and privacy
• ISSN: 2573-0142; 2573-0142
• DOI: 10.1145/3641015

Online harassment remains a prevalent problem for internet users. Its impact is made orders of magnitude worse when multiple harassers coordinate to conduct networked attacks. This paper presents an analysis of 231 threads in Kiwi Farms, a notorious online harassment community. We find that networked online harassment campaigns consists of three phases: target introduction, network decision, and network response. The first stage consists of the initial narrative elements, that are approved or not in stage two and expanded in stage three. Narrative building is a common element of all three stages. The network plays a key role in narrative building, adding elements to the narrative in at least 80 % of the threads, resulting in sustained harassment. This finding is central to our model of Continuous Narrative Escalation (CNE), that has two parts: (1) narrative continuation, the action of repeatedly adding new information to the existing narrative and (2) escalation, the aggravation of harassment that occurs as a consequence. In addition, we present insights from our analysis of 100 takedown requests threads, discussing received abuse reports. We find that these takedown requests are misused by the community and are used as elements to further fuel the narrative. We use our findings and framework to come up with a set of recommendations, that can inform harassment interventions and make online spaces safer.