When AI moderates online content: effects of human collaboration and interactive transparency on user trust

• Published in: Journal of computer-mediated communication 2022-07, Vol.27 (4)
• Main Authors: Molina, Maria D, Sundar, S Shyam
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial intelligence; Classification; User generated content
• ISSN: 1083-6101; 1083-6101
• DOI: 10.1093/jcmc/zmac010

Abstract Given the scale of user-generated content online, the use of artificial intelligence (AI) to flag problematic posts is inevitable, but users do not trust such automated moderation of content. We explore if (a) involving human moderators in the curation process and (b) affording “interactive transparency,” wherein users participate in curation, can promote appropriate reliance on AI. We test this through a 3 (Source: AI, Human, Both) × 3 (Transparency: No Transparency, Transparency-Only, Interactive Transparency) × 2 (Classification Decision: Flagged, Not Flagged) between-subjects online experiment (N = 676) involving classification of hate speech and suicidal ideation. We discovered that users trust AI for the moderation of content just as much as humans, but it depends on the heuristic that is triggered when they are told AI is the source of moderation. We also found that allowing users to provide feedback to the algorithm enhances trust by increasing user agency. Lay Summary As more users post in online forums, there has been a rise in harmful content such as hate speech and thoughts about committing suicide. Most online sites use humans to monitor such content. But, there is so much of this content each day that platforms have started using artificial intelligence (AI) to automatically flag and stop its spread. AI can be better than human moderators. It uses the same consistent criteria for classification, and it is faster. But, the problem is that people do not trust AI with such an important responsibility. One way to increase their trust is to involve humans in the moderation task. Another is to allow users to provide feedback on the classification. We conducted an experiment to test these ideas. Participants were told that the content was classified either by an AI, or by humans, or by both working together. Also, some participants were provided with a list of rules used for classification. Others were allowed to provide feedback about the rules. A third group did not receive any rules. We discovered that letting users provide feedback increased trust. Trust in AI also depends on the perceptions that users have about AI for moderating content.