A Hybrid Transformer Architecture for Multiclass Mental Illness Prediction using Social Media Text

Mental illness prediction through text involves employing natural language processing (NLP) techniques and deep learning algorithms to analyze textual data for the identification of mental disorders. Therefore, machine learning and deep learning algorithms have been utilized in the existing literatu...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2024-12, p.1-1
Main Authors: Karamat, Adnan, Imran, Muhammad, Yaseen, Muhammad Usman, Bukhsh, Rasool, Aslam, Sheraz, Ashraf, Nouman
Format: Article
Language:English
Subjects:
Anxiety disorders
convolutional neural network
Data models
Deep learning
Depression
Feature extraction
Linguistics
melBERT
Mental health
mental illness
MentalBERT
Predictive models
social media
Social networking (online)
transformer
Transformers
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Mental illness prediction through text involves employing natural language processing (NLP) techniques and deep learning algorithms to analyze textual data for the identification of mental disorders. Therefore, machine learning and deep learning algorithms have been utilized in the existing literature for the detection of mental illness. However, current systems exhibit suboptimal performance primarily due to their reliance on traditional embedding techniques and generic language models to generate text embeddings. To address this limitation, there is a requirement for domain-specific pretrained language models that comprehensively understand the context found in posts of mentally ill patients. Posts from individuals with mental illness often contain metaphorical expressions, posing a challenge for existing models in understanding such figurative language. In this study, we propose a hybrid transformer architecture, comprising MentalBERT and MelBERT pretrained language models, cascaded with CNN models to generate and concatenate deep features. MentalBERT is pretrained on an extensive corpus of text data specifically related to the mental health domain, while MelBERT is trained on a large corpus of metaphorical data for improved understanding of metaphorical expressions. The results reveal outstanding performance of the proposed architecture with an overall accuracy of 92% and an F1-score of 92%, surpassing state-of-the-art models in comparison. This study underscores the necessity for further research in this field and illustrates the potential of advanced technologies to address mental health issues in contemporary society.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2024.3519308