Enhancing Multimodal Sentiment Recognition Based on Cross-Modal Contrastive Learning

In recent years, multimodal sentiment recognition has gained attention for its potential to boost accuracy by combining information from various sources. Addressing the challenge of modality-based heterogeneity, we present Cross-Modal Contrastive Learning (CMCL), a novel framework. CMCL integrates d...

Full description

Saved in:
Bibliographic Details
Main Authors: Cui, Lichao, Yang, Shanliang
Format: Conference Proceeding
Language:English
Subjects:
Accuracy
Benchmark testing
Contrastive learning
Cross-modal Contrastive learning
Diversity reception
Multimodal fusion
Multimodal sentiment recognition
Visualization
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In recent years, multimodal sentiment recognition has gained attention for its potential to boost accuracy by combining information from various sources. Addressing the challenge of modality-based heterogeneity, we present Cross-Modal Contrastive Learning (CMCL), a novel framework. CMCL integrates diversity, consistency, and sample-level contrastive learning to enhance multimodal feature representation. Diversity contrastive learning separates modality-specific features into distinct spaces to capture their complementarity. Meanwhile, consistency contrastive learning aligns representations across modalities for consistency. Our approach outperforms existing baselines on three benchmark datasets, setting a new state-of-the-art standard.
ISSN:1945-788X
DOI:10.1109/ICME57554.2024.10688113