ESL students’ oral performance in English language school-based assessment: results of an empirical study

• Published in: Language Testing in Asia 2017-12, Vol.7 (1), p.1-21, Article 19
• Main Authors: Gan, Zhengdong, Oon, Emily Pey Tee, Davison, Chris
• Format: Article
• Language: English
• Subjects: Assessment; Education; Educational evaluation; English; English (Second Language); English as a second language instruction; Foreign Countries; Language assessment; Language Education; Language schools; Language Skills; Language Tests; Oral Language; Oral performance; Psychometrics; Rasch analysis; Rasch model; School-based assessment; Schools; Second Language Learning; Second language teachers; Secondary Education; Secondary School Students; Spoken language; Test Construction; Test Results; Testing and Evaluation
• ISSN: 2229-0443; 2229-0443
• DOI: 10.1186/s40468-017-0051-2

Background The English language school-based assessment (SBA) component of the Hong Kong Diploma of Secondary Education (HKDSE) Examination is innovative in that the assessment tasks involve assessing English oral language skills in a high-stakes context but they are designed and implemented in the ESL classroom by school teachers in light of a regular reading and viewing program or the elective modules integrated into the school curriculum. While this certainly is a positive move towards better congruence between the teaching, learning, and assessment activities, there has been concern whether the teachers are capable of applying the assessment criteria and standards consistently in spite of going through a variety of standardization meetings and sharing discussions initiated and mandated by the Hong Kong Examination and Assessment Authority (HKEAA). In other words, there has been concern about the extent to which results provided from teachers in different schools are comparable. Also, how may task difficulty be reflected in students’ assessment results across the two SBA task types? It was to provide some research evidence on matters relating to these issues associated with teacher assessment results that the study described here was carried out. Methods The study, with the help of Rasch analysis, aims to examine the psychometric qualities of this English language school-based assessment, how students’ assessment results may vary across different schools, and how task difficulty may vary across the two different task types. Results The findings indicated the following: (1) among the three schools involved in this study, two band 2 schools demonstrated similar abilities across all task domains as there were no significant differences in students’ SBA results in all assessment domains between these two band 2 schools. Significant differences were found in some assessment domains between the two band 2 schools and the band 3 school; (2) an obviously more fine-grained pattern of difference in difficulty levels of different assessment domains was observed in students’ assessment results across the two task types in this study than in previous studies. Conclusions Implications of the results for teacher assessor training and test task development are discussed.