Automatic classification of question turns in spontaneous speech using lexical and prosodic evidence

• Main Authors: Ananthakrishnan, Sankaranarayanan, Ghosh, Prasanta, Narayanan, Shrikanth
• Format: Conference Proceeding
• Language: English
• Subjects: Acoustic testing; Automatic speech recognition; Data mining; dialog; Error analysis; Humans; Natural languages; prosody; question turn; Robustness; speech act; Speech analysis; spontaneous speech; System testing
• ISSN: 1520-6149
• DOI: 10.1109/ICASSP.2008.4518782

The ability to identify speech acts reliably is desirable in any spoken language system that interacts with humans. Minimally, such a system should be capable of distinguishing between question-bearing turns and other types of utterances. However, this is a non-trivial task, since spontaneous speech tends to have incomplete syntactic, and even ungrammatical, structure and is characterized by disfluencies, repairs and other non-linguistic vocalizations that make simple rule based pattern learning difficult. In this paper, we present a system for identifying question-bearing turns in spontaneous multi-party speech (ICSI Meeting Corpus) using lexical and prosodic evidence. On a balanced test set, our system achieves an accuracy of 71.9% for the binary question vs. non-question classification task. Further, we investigate the robustness of our proposed technique to uncertainty in the lexical feature stream (e.g. caused by speech recognition errors). Our experiments indicate that classification accuracy of the proposed method is robust to errors in the text stream, dropping only about 0.8% for every 10% increase in word error rate (WER).