Self-Weighted Contrastive Fusion for Deep Multi-View Clustering

• Published in: IEEE transactions on multimedia 2024, Vol.26, p.9150-9162
• Main Authors: Wu, Song, Zheng, Yan, Ren, Yazhou, He, Jing, Pu, Xiaorong, Huang, Shudong, Hao, Zhifeng, He, Lifang
• Format: Article
• Language: English
• Subjects: deep clustering; Electronic mail; Matrix decomposition; multi-view clustering; Multi-view contrastive learning; representation degeneration; Representation learning; Self-supervised learning; Semantics; Symbols; Task analysis
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2024.3387298

Multi-view clustering can explore consensus information from multiple views and has attracted increasing attention in the past two decades. However, existing works face two major challenges: i) how to deal with the conflict between learning view-consensus information and reconstructing inconsistent view-private information and ii) how to mitigate representation degeneration caused by implementing the consistency objective for multi-view data. To address these challenges, we propose a novel framework of self-weighted contrastive fusion for deep multi-view clustering (SCMVC). First, our method establishes a hierarchical feature fusion framework, effectively segregating the consistency objective from the reconstruction objective. Then, multi-view contrastive fusion is implemented via maximizing consistency expression between the view-consensus representation and global representation, fully exploring the view consistency and complementary. More importantly, we propose to measure the discrepancy between pairwise representations, and then introduce a self-weighting method, which adaptively strengthens useful views in feature fusion and weakens unreliable views, to mitigate representation degeneration. Extensive experiments on nine public datasets demonstrate that our proposed method achieves state-of-the-art clustering performance.