An investigation of the validity of a speaking assessment for adolescent English language learners

• Published in: Language testing 2021-07, Vol.38 (3), p.401-428
• Main Authors: Huang, Becky H., Bailey, Alison L., Sass, Daniel A., Shawn Chang, Yung-hsiang
• Format: Article
• Language: English
• Subjects: Case Studies; Communicative competence; Construct Validity; Correlation; Educational Testing; English (Second Language); English as a second language; English as a second language instruction; English as a second language tests; English language learners; Factor Analysis; Foreign Countries; Grade 7; Grade 8; Grade 9; Kindergarten; Language Skills; Language Tests; Learning Experience; Second Language Instruction; Second Language Learning; Secondary school students; Self Evaluation (Individuals); Speech Communication; Spoken language; Student Attitudes; Teenagers; Test Validity; Validation studies; Validity
• ISSN: 0265-5322; 1477-0946
• DOI: 10.1177/0265532220925731

Given the increasing emphasis of communicative competence in English as a foreign language (EFL) contexts and the lack of validation research on speaking assessments for adolescent EFL learners, in the current study we examined the validity of the TOEFL Junior® speaking test, a relatively new speaking assessment developed by Educational Testing Service. We utilized Messick’s integrative view of construct validity (1996) and Kane’s interpretation argument framework (Kane, 2013), and focused on two sources of validity evidence: internal structure and external relations to other variables (AERA, APA, and NCME, 2014). We used a cross-sectional design with 252 seventh-to-ninth graders in Taiwan, which aligns with the spectrum of the TOEFL Junior®’s target age group. All adolescent participants took the TOEFL Junior® speaking test and two researcher-developed speaking tests. They also completed a survey reporting their history of English learning experiences from kindergarten to present and evaluating their own English language skills. Adolescent participants’ teachers provided evaluations of participants’ English language skills. Results from correlational tests and confirmatory factor analysis showed robust evidence for the test’s internal structure. Correlation and multiple regression models also demonstrated strong positive relationships between TOEFL Junior® test scores and external variables, providing evidence for TOEFL Junior®’s construct validity.