Using articulatory feature detectors in progressive networks for multilingual low-resource phone recognition

• Published in: The Journal of the Acoustical Society of America 2024-11, Vol.156 (5), p.3411-3421
• Main Authors: Morshed, Mahir, Hasegawa-Johnson, Mark
• Format: Article
• Language: English
• ISSN: 0001-4966
• DOI: 10.1121/10.0034415

Systems inspired by progressive neural networks, transferring information from end-to-end articulatory feature detectors to similarly structured phone recognizers, are described. These networks, connecting the corresponding recurrent layers of pre-trained feature detector stacks and newly introduced phone recognizer stacks, were trained on data from four Asian languages, with experiments testing the system on those languages and four African languages. Later adjustments of these networks include the use of contrastive predictive coding layers at the inputs to those networks' recurrent portions. Such adjustments allow for performance differences to be attributed to the presence or absence of individual feature detectors (for consonant place/manner and vowel height/backness). Some of these differences manifest after feature-level comparisons of recognizer outputs, as well as through considering variations and ablations in architecture and training setup. These differences encourage further exploration of methods to reduce errors with phones having specific articulatory features as well as further architectural modifications.