Computer-assisted assessment of phonetic fluency in a second language: a longitudinal study of Japanese learners of French

• Published in: Speech communication 2020-12, Vol.125, p.69-79
• Main Authors: Detey, Sylvain, Fontan, Lionel, Le Coz, Maxime, Jmel, Saïd
• Format: Article
• Language: English
• Subjects: Acoustic phonetics; Algorithms; Assessment; Automatic; Automatic speech recognition; Colleges & universities; Computerized corpora; Corpus linguistics; Correlation analysis; Correlation coefficients; Evolutionary algorithms; Fluency; French; French as a second language; Japanese; Japanese language; Longitudinal; Longitudinal studies; Perceptions; Phonetics; Pronunciation instruction; Questions; Ratings; Regression models; Regularity; Second language learning; Segmentation; Sentences; Smoothness; Speech; Speech processing; Speech rate; Speech recognition; Viscosity; Voice recognition
• ISSN: 0167-6393; 1872-7182
• DOI: 10.1016/j.specom.2020.10.001

•Automatic assessment of L2 phonetic fluency in read-aloud speech without an ASR system.•High agreement between automatic and human assessments.•Longitudinal coherence between fluency scores and learning stage. Automatic second language (L2) speech fluency assessment has been one of the ultimate goals of several projects aiming at designing Computer-Assisted Pronunciation Training (CAPT) tools for L2 learners. Usually, three challenges must be tackled in order to solve the issues at stake: 1) Defining fluency from a threefold interdisciplinary perspective (acoustic and perceptual phonetics, computer science, L2 education); 2) Using a cost-effective algorithm; 3) Testing the procedure with actual learners’ data. Despite rapid technical developments in the field of automatic speech processing, the tools which are actually available for learners are still scarce, and most of them rely on automatic speech recognition (ASR). Moreover, most research on the topic is focusing on English as the target L2. Therefore, in this article, we address the following research questions: (a) is it possible to use a non-ASR-based low-level signal segmentation algorithm to predict human expert assessment of phonetic fluency in beginner Japanese learners of French in a text-reading task during the first stages of their learning? (b) if the answer to (a) is positive, then what are the best predictors of phonetic fluency among a set of available measures (see below for more details)? (c) is it possible to use this algorithm to monitor the evolution of phonetic fluency (and of its associated predictors) in these learners in a longitudinal study? As a first step, a corpus of French sentences read aloud by 12 Japanese learners of different proficiency levels in French was used to design a prediction system. The read-aloud speech data was perceptually annotated by three human experts on four dimensions: overall speech fluency, speech rate, regularity of speech rate, speech fluidity (i.e. smoothness of transitions between phones). Inter-rater agreement and reliability were high for all dimensions, and the average human ratings were compared with the scores provided by our prediction system. The results show strong correlations between human and automatic scores of speech rate and regularity of speech rate, and a weak correlation for speech fluidity. Automatic scores were finally combined together through a multiple linear regression model in order to predict overall speech fluency. The