State Graph Reasoning for Multimodal Conversational Recommendation

Conversational recommendation system (CRS) attracts increasing attention in various application domains such as retail and travel. It offers an effective way to capture users' dynamic preferences with multi-turn conversations. However, most current studies center on the recommendation aspect wh...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on multimedia 2023, Vol.25, p.3113-3124
Main Authors: Wu, Yuxia, Liao, Lizi, Zhang, Gangyi, Lei, Wenqiang, Zhao, Guoshuai, Qian, Xueming, Chua, Tat-Seng
Format: Article
Language:English
Subjects:
Cognition
Conversation
Data structures
Decision making
Electronic mail
History
knowledge graph
Knowledge representation
Negligence
Reasoning
recommend- ation systems
Recommender systems
Training
User requirements
Visualization
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Conversational recommendation system (CRS) attracts increasing attention in various application domains such as retail and travel. It offers an effective way to capture users' dynamic preferences with multi-turn conversations. However, most current studies center on the recommendation aspect while over-simplifying the conversation process. The negligence of complexity in data structure and conversation flow hinders their practicality and utility. In reality, there exist various relationships among slots and values, while users' requirements may dynamically adjust or change. Moreover, the conversation often involves visual modality to facilitate the conversation. These actually call for a more advanced internal state representation of the dialogue and a proper reasoning scheme to guide the decision making process. In this paper, we explore multiple facets of multimodal conversational recommendation and try to address the above mentioned challenges. In particular, we represent the structured back-end database as a multimodal knowledge graph which captures the various relations and evidence in different modalities. The user preferences expressed via conversation utterances will then be gradually updated to the state graph with clear polarity. Based on these, we train an end-to-end S tate G raph-based R easoning model ( SGR ) to perform reasoning over the whole state graph. The prediction of our proposed model benefits from the structure of the graph. It not only allows for zero-shot reasoning for items unseen in training conversations, but also provides a natural way to explain the policies. Extensive experiments show that our model achieves better performance compared with existing methods.
ISSN:1520-9210
1941-0077
DOI:10.1109/TMM.2022.3155900