Information fusion for mental disorders detection: multimodal BERT against fusioning multiple BERTs

Given the increasing number of modalities that modern classification problems provide, recently a multimodal BERT transformer (MMBT) was proposed. An interesting opportunity to evaluate the effectiveness of such model is posed by the problem of timely detection of mental disorders of social media us...

Full description

Saved in:
Bibliographic Details
Published in:Procesamiento del Lenguaje Natural 2022-09, Vol.69, p.27
Main Authors: Aragón, Mario Ezra, A. Pastor López-Monroy, González-Gurrola, Luis C, Montes-y-Gómez, Manuel
Format: Article
Language:English
Subjects:
Classification
Data integration
Mental disorders
Multimodality
Natural language processing
Social media
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Given the increasing number of modalities that modern classification problems provide, recently a multimodal BERT transformer (MMBT) was proposed. An interesting opportunity to evaluate the effectiveness of such model is posed by the problem of timely detection of mental disorders of social media users. For this problem, a multi-channel perspective involves extracting from each user post different types of information, such as thematic, emotional and stylistic content. This study evaluates the suitability of tackling this problem by the apparently ad-hoc MMBT, moreover, we further evaluate if regular BERT models could be combined or fused in such a way that could have a chance in a multi-channel arena. For the evaluation, we use recent public data sets for three important mental disorders: Depression, Anorexia, and Self-harm. Results suggest that BERT models can get on their own a data representation that could be later fusioned and boost the classification performance by at least 5% in F1 measure, even surpassing the MMBT.
ISSN:1135-5948
1989-7553
DOI:10.26342/2022-69-2