Towards a Universal NLG for Dialogue Systems and Simulators with Future Bridging

In a dialogue system pipeline, a natural language generation (NLG) unit converts the dialogue direction and content to a corresponding natural language realization. A recent trend for dialogue systems is to first pre-train on large datasets and then fine-tune in a supervised manner using datasets an...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2021-05
Main Authors: Ennen, Philipp, Yen-Ting, Lin, Ozbay, Ali Girayhan, Insalata, Ferdinando, Li, Maolin, Tian, Ye, Jalali, Sepehr, Da-shan, Shiu
Format: Article
Language:English
Subjects:
Annotations
Datasets
Interactive computer systems
Massive data points
Natural language
Simulators
Training
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In a dialogue system pipeline, a natural language generation (NLG) unit converts the dialogue direction and content to a corresponding natural language realization. A recent trend for dialogue systems is to first pre-train on large datasets and then fine-tune in a supervised manner using datasets annotated with application-specific features. Though novel behaviours can be learned from custom annotation, the required effort severely bounds the quantity of the training set, and the application-specific nature limits the reuse. In light of the recent success of data-driven approaches, we propose the novel future bridging NLG (FBNLG) concept for dialogue systems and simulators. The critical step is for an FBNLG to accept a future user or system utterance to bridge the present context towards. Future bridging enables self supervised training over annotation-free datasets, decoupled the training of NLG from the rest of the system. An FBNLG, pre-trained with massive datasets, is expected to apply in classical or new dialogue scenarios with minimal adaptation effort. We evaluate a prototype FBNLG to show that future bridging can be a viable approach to a universal few-shot NLG for task-oriented and chit-chat dialogues.
ISSN:2331-8422