MS²-GNN: Exploring GNN-Based Multimodal Fusion Network for Depression Detection

Major depressive disorder (MDD) is one of the most common and severe mental illnesses, posing a huge burden on society and families. Recently, some multimodal methods have been proposed to learn a multimodal embedding for MDD detection and achieved promising performance. However, these methods ignor...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on cybernetics 2023-12, Vol.53 (12), p.7749-7759
Main Authors: Chen, Tao, Hong, Richang, Guo, Yanrong, Hao, Shijie, Hu, Bin
Format: Article
Language:English
Subjects:
Algorithms
Graph neural networks
Heterogeneity
Homogeneity
Mental depression
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Major depressive disorder (MDD) is one of the most common and severe mental illnesses, posing a huge burden on society and families. Recently, some multimodal methods have been proposed to learn a multimodal embedding for MDD detection and achieved promising performance. However, these methods ignore the heterogeneity/homogeneity among various modalities. Besides, earlier attempts ignore interclass separability and intraclass compactness. Inspired by the above observations, we propose a graph neural network (GNN)-based multimodal fusion strategy named modal-shared modal-specific GNN, which investigates the heterogeneity/homogeneity among various psychophysiological modalities as well as explores the potential relationship between subjects. Specifically, we develop a modal-shared and modal-specific GNN architecture to extract the inter/intramodal characteristics. Furthermore, a reconstruction network is employed to ensure fidelity within the individual modality. Moreover, we impose an attention mechanism on various embeddings to obtain a multimodal compact representation for the subsequent MDD detection task. We conduct extensive experiments on two public depression datasets and the favorable results demonstrate the effectiveness of the proposed algorithm.
ISSN:2168-2267
2168-2275
DOI:10.1109/TCYB.2022.3197127