Image-text interaction graph neural network for image-text sentiment analysis

As various social platforms are experiencing fast development, the volume of image-text content generated by users has grown rapidly. Image-text based sentiment of social media analysis has also attracted great interest from researchers in recent years. The main challenge of image-text sentiment ana...

Full description

Saved in:
Bibliographic Details
Published in:Applied intelligence (Dordrecht, Netherlands) Netherlands), 2022-08, Vol.52 (10), p.11184-11198
Main Authors: Liao, Wenxiong, Zeng, Bi, Liu, Jianqi, Wei, Pengfei, Fang, Jiongkun
Format: Article
Language:English
Subjects:
Artificial Intelligence
Artificial neural networks
Computer Science
Data mining
Feature extraction
Graph neural networks
Image classification
Machines
Manufacturing
Mechanical Engineering
Neural networks
Processes
Sentiment analysis
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:As various social platforms are experiencing fast development, the volume of image-text content generated by users has grown rapidly. Image-text based sentiment of social media analysis has also attracted great interest from researchers in recent years. The main challenge of image-text sentiment analysis is how to construct a model that can promote the complementarity between image and text. In most previous studies, images and text were simply merged, while the interaction between them was not fully considered. This paper proposes an image-text interaction graph neural network for image-text sentiment analysis. A text-level graph neural network is used to extract the text features, and a pre-trained convolutional neural network is employed to extract the image features. Then, an image-text interaction graph network is constructed. The node features of the graph network are initialized by the text features and the image features, while the node features in the graph are updated based on the graph attention mechanism. Finally, combined with image-text aggregation layer to realize sentiment classification. The results of the experiments prove that the presented method is more effective than existing methods. In addition, a large-scale Twitter image-text sentiment analysis dataset was built by us and used in the experiments.
ISSN:0924-669X
1573-7497
DOI:10.1007/s10489-021-02936-9