Convergent evidence for validity of a performance-based ICT skills test

The goal of this study was to investigate sources of evidence of convergent validity supporting the construct interpretation of scores on a simulation-based ICT skills test. The construct definition understands ICT skills as reliant on ICT-specific knowledge as well as comprehension and problem-solv...

Full description

Saved in:
Bibliographic Details
Published in:European journal of psychological assessment : official organ of the European Association of Psychological Assessment 2020, Vol.36 (2), p.269-279
Main Authors: Engelhardt, Lena, Naumann, Johannes, Goldhammer, Frank, Frey, Andreas, Wenzel, S. Franziska C, Hartig, Katja, Horz, Holger
Format: Article
Language:English
Subjects:
Deutschland
Evidenz
Fertigkeit
Informations- und Kommunikationstechnologie
Informationstechnologische Bildung
Kompetenz
Problemlösen
Schüler
Sekundarstufe I
Test
Testaufgabe
Validität
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The goal of this study was to investigate sources of evidence of convergent validity supporting the construct interpretation of scores on a simulation-based ICT skills test. The construct definition understands ICT skills as reliant on ICT-specific knowledge as well as comprehension and problem-solving skills. On the basis of this, a validity argument comprising three claims was formulated and tested. (1) In line with the classical nomothetic span approach, all three predictor variables explained task success positively across all ICT skills items. As ICT tasks can vary in the extent to which they require construct-related knowledge and skills and in the way related items are designed and implemented, the effects of construct-related predictor variables were expected to vary across items. (2) A task-based analysis approach revealed that the item-level effects of the three predictor variables were in line with the targeted construct interpretation for most items. (3) Finally, item characteristics could significantly explain the random effect of problem-solving skills, but not comprehension skills. Taken together, the obtained results generally support the validity of the construct interpretation.
ISSN:1015-5759
2151-2426
DOI:10.1027/1015-5759/a000507