Knowledge Distillation for Improved Accuracy in Spoken Question Answering

Spoken question answering (SQA) is a challenging task that requires the machine to fully understand the complex spoken documents. Automatic speech recognition (ASR) plays a significant role in the development of QA systems. However, the recent work shows that ASR systems generate highly noisy transc...

Full description

Saved in:
Bibliographic Details
Main Authors: You, Chenyu, Chen, Nuo, Zou, Yuexian
Format: Conference Proceeding
Language:English
Subjects:
Conferences
Knowledge discovery
knowledge distillation
Manuals
Natural language processing
question answering
Signal processing
spoken question answering
Syntactics
Training
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Spoken question answering (SQA) is a challenging task that requires the machine to fully understand the complex spoken documents. Automatic speech recognition (ASR) plays a significant role in the development of QA systems. However, the recent work shows that ASR systems generate highly noisy transcripts, which critically limit the capability of machine comprehension on the SQA task. To address the issue, we present a novel distillation framework. Specifically, we devise a training strategy to perform knowledge distillation (KD) from spoken documents and written counterparts. Our work aims at distilling rich knowledge from the language model to improve the performance of the student model by reducing the misalignment between automatic and manual transcripts. Experiments demonstrate that our approach outperforms several state-of-the-art language models on the Spoken-SQuAD dataset.
ISSN:2379-190X
DOI:10.1109/ICASSP39728.2021.9414999