RUN-AS: a novel approach to annotate news reliability for disinformation detection

The development of the internet and digital technologies has inadvertently facilitated the huge disinformation problem that faces society nowadays. This phenomenon impacts ideologies, politics and public health. The 2016 US presidential elections, the Brexit referendum, the COVID-19 pandemic and the...

Full description

Saved in:
Bibliographic Details
Published in:Language resources and evaluation 2024-06, Vol.58 (2), p.609-639
Main Authors: Bonet-Jover, Alba, Sepúlveda-Torres, Robiert, Saquete, Estela, Martínez-Barco, Patricio, Nieto-Pérez, Mario
Format: Article
Language:English
Subjects:
Algorithms
Annotations
Computational Linguistics
Computer Science
Datasets
Decision trees
Deep learning
False information
Internet
Language and Literature
Linguistics
Machine learning
News
News media
Original Paper
Presidential elections
Public health
Reliability
Social Sciences
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The development of the internet and digital technologies has inadvertently facilitated the huge disinformation problem that faces society nowadays. This phenomenon impacts ideologies, politics and public health. The 2016 US presidential elections, the Brexit referendum, the COVID-19 pandemic and the Russia-Ukraine war have been ideal scenarios for the spreading of fake news and hoaxes, due to the massive dissemination of information. Assuming that fake news mixes reliable and unreliable information, we propose RUN-AS (Reliable and Unreliable Annotation Scheme), a fine-grained annotation scheme that enables the labelling of the structural parts and essential content elements of a news item and their classification into Reliable and Unreliable. This annotation proposal aims to detect disinformation patterns in text and to classify the global reliability of news. To this end, a dataset in Spanish was built and manually annotated with RUN-AS and several experiments using this dataset were conducted to validate the annotation scheme by using Machine Learning (ML) and Deep Learning (DL) algorithms. The experiments evidence the validity of the annotation scheme proposed, obtaining the best F 1 m , 0.948, with the Decision Tree algorithm.
ISSN:1574-020X
1574-0218
DOI:10.1007/s10579-023-09678-9