The inter-rater reliability of the story retell procedure

• Published in: Aphasiology 2003-05, Vol.17 (5), p.523-528
• Main Authors: Hula, William, McNeil, Malcolm, Doyle, Patrick, Rubinsky, Hillel, Fossett, Tepanta
• Format: Article
• Language: English
• ISSN: 0268-7038; 1464-5041
• DOI: 10.1080/02687030344000139

Background: McNeil, Doyle, Fossett, Park, and Goda (2001) have presented the story retell procedure (SRP) as an efficient means of assessing discourse in adults with aphasia, in part because it provides reliable, valid, and sensitive indices of performance without the need for time-consuming transcription of language samples. Aims: The purpose of this study was to demonstrate that the SRP, when scored without transcription by judges with minimal training, produces a reliable measure of information transfer. Methods & Procedures: Four judges who had not used the SRP previously scored audio-recorded language samples, produced by four subjects with aphasia and eleven normal subjects, for percent information units per minute (%IU/Min). Outcomes & Results: The results demonstrate that the SRP has high inter-rater reliability. Reliability coefficients ranged from .89 to .995, and the standard error of measurement associated with inter-rater scoring error ranged from .59 to 1.42 %IU/Min. Point-to-point reliability in scoring individual information units ranged from 85-95% and averaged 91% for both subject groups. Conclusions: The SRP is a potentially useful tool for quantifying connected language behaviour, and may be particularly valuable in clinical and research settings where economy of assessment procedures is essential.