Autoregressive Predictive Coding: A Comprehensive Study

• Published in: IEEE journal of selected topics in signal processing 2022-10, Vol.16 (6), p.1380-1390
• Main Authors: Yang, Gene-Ping, Yeh, Sung-Lin, Chung, Yu-An, Glass, James, Tang, Hao
• Format: Article
• Language: English
• Subjects: Algorithms; Automatic speech recognition; Autoregressive processes; Classification; Coding; Fundamental frequency; Linguistics; Phonetics; Predictive coding; Prosody; Representation learning; Representations; Resonant frequencies; Self-supervised learning; Speaker identification; speaker verification; Speech recognition; Voice recognition
• ISSN: 1932-4553; 1941-0484
• DOI: 10.1109/JSTSP.2022.3203608

We review autoregressive predictive coding (APC), an approach to learn speech representation by predicting a future frame given the past frames. We present three different views of interpreting APC, and provide a historical account to the approach. To study the speech representation learned by APC, we use common speech tasks, such as automatic speech recognition and speaker verification, to demonstrate the utility of the learned representation. In addition, we design a suite of fine-grained tasks, including frame classification, segment classification, fundamental frequency tracking, and duration prediction, to probe the phonetic and prosodic content of the representation. The three views of the APC objective welcome various generalizations and algorithms to learn speech representations. Probing on the suite of fine-grained tasks suggests that APC makes a wide range of high-level speech information accessible in its learned representation.