Reliability and validity of a self-administered tool for online neuropsychological testing: The Amsterdam Cognition Scan

• Published in: Journal of clinical and experimental neuropsychology 2018-04, Vol.40 (3), p.253-273
• Main Authors: Feenstra, Heleen E. M., Murre, Jaap M. J., Vermeulen, Ivar E., Kieffer, Jacobien M., Schagen, Sanne B.
• Format: Article
• Language: English
• Subjects: Amsterdam Cognition Scan; Cancer; Cognition & reasoning; Cognitive ability; computer skills; concurrent validity; Internet; intraclass correlation coefficient; Measuring techniques; neuropsychological test battery; Neuropsychology; oncology; Psychology; Spatial memory; test development; test-retest reliability; unmonitored setting; Validity
• ISSN: 1380-3395; 1744-411X
• DOI: 10.1080/13803395.2017.1339017

Introduction: To facilitate large-scale assessment of a variety of cognitive abilities in clinical studies, we developed a self-administered online neuropsychological test battery: the Amsterdam Cognition Scan (ACS). The current studies evaluate in a group of adult cancer patients: test-retest reliability of the ACS and the influence of test setting (home or hospital), and the relationship between our online and a traditional test battery (concurrent validity). Method: Test-retest reliability was studied in 96 cancer patients (57 female; M age  = 51.8 years) who completed the ACS twice. Intraclass correlation coefficients (ICCs) were used to assess consistency over time. The test setting was counterbalanced between home and hospital; influence on test performance was assessed by repeated measures analyses of variance. Concurrent validity was studied in 201 cancer patients (112 female; M age  = 53.5 years) who completed both the online and an equivalent traditional neuropsychological test battery. Spearman or Pearson correlations were used to assess consistency between online and traditional tests. Results: ICCs of the online tests ranged from .29 to .76, with an ICC of .78 for the ACS total score. These correlations are generally comparable with the test-retest correlations of the traditional tests as reported in the literature. Correlating online and traditional test scores, we observed medium to large concurrent validity (r/ρ = .42 to .70; total score r = .78), except for a visuospatial memory test (ρ = .36). Correlations were affected-as expected-by design differences between online tests and their offline counterparts. Conclusions: Although development and optimization of the ACS is an ongoing process, and reliability can be optimized for several tests, our results indicate that it is a highly usable tool to obtain (online) measures of various cognitive abilities. The ACS is expected to facilitate efficient gathering of data on cognitive functioning in the near future.