Automatic Detection of Offensive Language for Urdu and Roman Urdu

In recent years, unethical behavior in the cyber-environment has been revealed. The presence of offensive language on social media platforms and automatic detection of such language is becoming a major challenge in modern society. The complexity of natural language constructs makes this task even mo...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2020, Vol.8, p.91213-91226
Main Authors: Akhter, Muhammad Pervez, Jiangbin, Zheng, Naqvi, Irfan Raza, Abdelmajeed, Mohammed, Sadiq, Muhammad Tariq
Format: Article
Language:English
Subjects:
Datasets
Digital media
English language
Feature extraction
Language
Languages
Machine learning
natural language Processing
offensive language detection
Scripts
Social media
Social networks
text processing
Twitter
Urdu language
User generated content
Videos
Words (language)
Writing
YouTube
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In recent years, unethical behavior in the cyber-environment has been revealed. The presence of offensive language on social media platforms and automatic detection of such language is becoming a major challenge in modern society. The complexity of natural language constructs makes this task even more challenging. Until now, most of the research has focused on resource-rich languages like English. Roman Urdu and Urdu are two scripts of writing the Urdu language on social media. The Roman script uses the English language characters while the Urdu script uses Urdu language characters. Urdu and Hindi languages are similar with the only difference in their writing script but the Roman scripts of both languages are similar. This study is about the detection of offensive language from the user's comments presented in a resource-poor language Urdu. We propose the first offensive dataset of Urdu containing user-generated comments from social media. We use individual and combined n-grams techniques to extract features at character-level and word-level. We apply seventeen classifiers from seven machine learning techniques to detect offensive language from both Urdu and Roman Urdu text comments. Experiments show that the regression-based models using character n-grams show superior performance to process the Urdu language. Character-level tri-gram outperforms the other word and character n-grams. LogitBoost and SimpleLogistic outperform the other models and achieve 99.2% and 95.9% values of F-measure on Roman Urdu and Urdu datasets respectively. Our designed dataset is publically available on GitHub for future research.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2020.2994950