Validity, reliability, and utility of the Observation Survey of Early Literacy Achievement

• Published in: Reading research quarterly 2006-01, Vol.41 (1), p.8-34
• Main Authors: DENTON, CAROLYN A., CIANCIO, DENNIS J., FLETCHER, JACK M.
• Format: Article
• Language: English
• Subjects: assessment; authentic; Benchmarks; Children; Correlations; criterion; diagnostic; Diagnostic tests; Dictation; early childhood; Early literacy; Early Reading; Educational administration; Educational research; Educational sciences; Elementary education; Elementary school students; Emergent Literacy; Evaluation Methods; Evaluation Research; Grade 1; Hearing; inform; informal; Intervention; learner; Learning; Letter recognition; Literacy programs; Measurement Techniques; Methodology; Observation; Observational research; ongoing; outcomes; Polls & surveys; Predictive Validity; Preschool Children; program; Program Evaluation; Progress Monitoring; Psychometrics; Reading achievement; Reading difficulties; Reading Failure; Reading instruction; Reading programs; Reading Skills; Records & achievements; screening; standardized; Standardized tests; strategies; Student Evaluation; Subtests; Test Reliability; Test Validity; topic; type; Vocabulary; Word recognition; Writing
• ISSN: 0034-0553; 1936-2722
• DOI: 10.1598/RRQ.41.1.1

This Study evaluated the validity, reliability, and utility of five of the six subtests of Clay's (2002) Observation Survey of Early Literacy Achievement (OS), including Letter Identification, Word Reading, Writing Vocabulary, Hearing and Recording Sounds in Words, and Text Reading. In addition to a review of studies providing assessment of the reliability and validity of the OS, we analyzed (a) the concurrent validity of the OS subtests relative to other instruments with known psychometric properties, (b) their validity and accuracy as predictors of reading development in grade 1, and (c) interadministrator reliability for the Text Reading and Writing Vocabulary subtests. The score distributions of most of the OS subtests indicated inadequate floors and ceilings, suggesting that when the OS subtests are scored as recommended in the manual, they are not suitable for program evaluation. Analyses of predictive validity indicated that decisions based on the attainment of established benchmarks in OS Word Identification, Text Reading, and Writing Vocabulary would be valid for early screening purposes and to evaluate student outcomes, but more work needs to be done in setting benchmarks for these subtests. In addition to this evidence for predictive validity, there was evidence for concurrent validity of the OS. Interassessor reliability was high for both the Text Reading and Writing Vocabulary subtests in our sample. Our review of the characteristics and administration procedures of each subtest suggests cautions regarding the utility of some of the measures for diagnostic assessment and progress monitoring. Overall, with some limitations, the OS can be validly implemented to assess components of early reading development. /// [Spanish] Este estudio evaluó la validez, confiabilidad y utilidad de cinco de los seis subtests del Informe de Observación de la Alfabetización Temprana (0S) de Clay (2002). Los tests incluyen Identificación de Letras, Lectura de Palabras, Vocabulario Escrito, Escuchar y Grabar Sonidos y Lectura de Textos. Además de revisar estudios que evalúan la confiabilidad y validez del OS, analizamos: (a) la validez concurrente de los subtests del OS en relación a otros instrumentos de conocidas propiedades psicométricas, (b) la validez y precisión como predictores del desarrollo lector en l° Grado y (c) la validez inter-evaluador en los tests de Lectura de Texros y Vocabulario Escrito. Las distribuciones de los valores de la mayoría de los subtet