In-house or commercial speaking tests: Evaluating strengths for EAP placement

• Published in: Journal of English for academic purposes 2013-12, Vol.12 (4), p.288-298
• Main Authors: Jamieson, Joan, Wang, Linxiao, Church, Jacqueline
• Format: Article
• Language: English
• Subjects: Assessment; Computers; ESL; Speaking; Versant English
• ISSN: 1475-1585; 1878-1497
• DOI: 10.1016/j.jeap.2013.09.003

When language program administrators consider changing a placement test, there are many issues to address. Will the scores help us place students into our curriculum? Will the scores reflect real differences in students' abilities? Will the administration of the test be feasible? This article describes one program's deliberations between keeping an in-house test or adopting a commercial test for speaking. Two speaking tests were compared according to curricular coverage, statistical distributions, and practicality. One test, PIE Speaking, was developed in-house. The other test, Versant English, was developed by Pearson Knowledge Technologies. Both covered many but not all curricular objectives. Internal consistency estimates were higher for Versant English than for PIE Speaking. The comparison of distribution patterns suggested that PIE Speaking better discriminated between mid-level students, but Versant English better discriminated between low and high ability students. PIE Speaking took approximately 60 staff hours, costing about $1200. Versant English took about 10 staff hours at an estimated cost of $6500. Cost weighed most heavily in the decision to keep the in-house speaking test. Modeling the steps taken to answer specific questions may provide structure for other language programs when evaluating their placement tests. •Two speaking tests compared to determine appropriateness as basis for placement decisions.•One test, PIE Speaking, was developed in-house; the other, Versant English, a commercial product.•PIE Speaking was better at distinguishing students at intermediate levels, Versant English at high levels.•In terms of time, the fully automated computerized test, Versant English, was more efficient.•The in-house test, PIE Speaking, was much, much less expensive in terms of financial costs.