COTA 2.0 : an automatic corrector of Tunisian Arabic social media texts

In written text, orthographic noise is a common concern for NLP, especially when operating social-network comments and raw documents. This is mainly due to its orthographic conventions and morphological ambiguity. We propose to automatically normalize the social-media dialect corpora by following CO...

Full description

Saved in:
Bibliographic Details
Published in:Jordanian Journal of Computers and Information Technology 2022-12, Vol.8 (4), p.370-387
Main Authors: al-Lawzi, Maryam, Zribi, Inès, Balghayth, Lamya Hadrich, Makki, Asma
Format: Article
Language:ara ; eng
Subjects:
Algorithms
coda-tun
Conventions
cota orthography system
Dialects
Error correction & detection
Information technology
Language
Linguistics
Morphology
orthographic normalization
Orthography
Social networks
Spelling
tunisian arabic
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In written text, orthographic noise is a common concern for NLP, especially when operating social-network comments and raw documents. This is mainly due to its orthographic conventions and morphological ambiguity. We propose to automatically normalize the social-media dialect corpora by following CODA-TA, the conventional Orthography for TA. The existing system developed for TA «COTA Orthography 1.0» is not able to handle all forms of TA. Therefore, we propose to extend its rules and lexicons to address the peculiarities of social media dialect. In certain words, the COTA Orthography 1.0 system provides the user with several correction possibilities. Therefore, in the new version, we incorporated a trigram language model to automatically select the right correction. Our results show that the system can reduce transcription errors by 95.72%.
ISSN:2413-9351
2415-1076
DOI:10.5455/jjcit.71-1655499240