DFM: Dialogue Foundation Model for Universal Large-Scale Dialogue-Oriented Task Learning

Building a universal conversational agent has been a long-standing goal of the dialogue research community. Most previous works only focus on a small set of dialogue tasks. In this work, we aim to build a unified dialogue foundation model (DFM) which can be used to solve massive diverse dialogue tas...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2022-10
Main Authors: Chen, Zhi, Bao, Jijia, Chen, Lu, Liu, Yuncong, Ma, Da, Chen, Bei, Wu, Mengyue, Zhu, Su, Dong, Xin, Ge, Fujiang, Miao, Qingliang, Jian-Guang Lou, Yu, Kai
Format: Article
Language:English
Subjects:
Datasets
Distillation
Downstream effects
Skills
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Building a universal conversational agent has been a long-standing goal of the dialogue research community. Most previous works only focus on a small set of dialogue tasks. In this work, we aim to build a unified dialogue foundation model (DFM) which can be used to solve massive diverse dialogue tasks. To achieve this goal, a large-scale well-annotated dialogue dataset with rich task diversity (DialogZoo) is collected. We introduce a framework to unify all dialogue tasks and propose novel auxiliary self-supervised tasks to achieve stable training of DFM on the highly diverse large scale DialogZoo corpus. Experiments show that, compared with models of the same size, DFM can achieve state-of-the-art or competitive performance on very rich cross-domain downstream dialogue tasks. This demonstrates that DFM largely extends the ability of unified dialogue pre-trained model.
ISSN:2331-8422