Measure for Measure: A two study multi-trait multi-method investigation of construct validity in IS research

► Results generally support a correspondence between self-reported and neurophysiological measures. ► Primitive perceptual constructs are not affected by mono-method bias. ► Complex perceptual measures are more likely to suffer from mono-method bias. ► Within method correlations can be due to comple...

Full description

Saved in:
Bibliographic Details
Published in:Computers in human behavior 2013-05, Vol.29 (3), p.833-844
Main Authors: Ortiz de Guinea, Ana, Titah, Ryad, Léger, Pierre-Majorique
Format: Article
Language:English
Subjects:
Construct validity
Electroencephalography
Instrument validation
Mono method bias
Multi-trait multi-method
Neuro IS
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:► Results generally support a correspondence between self-reported and neurophysiological measures. ► Primitive perceptual constructs are not affected by mono-method bias. ► Complex perceptual measures are more likely to suffer from mono-method bias. ► Within method correlations can be due to complexity of trait method and congeneric method effects. Given the importance and criticality of instrument validation in IS research, the main objective of this study is to provide a systematic assessment of IS construct validity via multi-trait multi-method (MTMM) matrix. To do so, the paper uses structurally different methods – neurophysiological and self-reported instruments – to measure three important and commonly used IS constructs: engagement, arousal and cognitive load in two different experimental settings. The experiments involved seventeen (17) and twenty-four (24) participants respectively and consisted in using different IS to execute a set of both instrumental and hedonic tasks. The results generally support MTMM matrix expectations and shed light on the complexity of detecting the nature of mono-method bias. Specifically, the results show that primitive perceptual IS constructs such as arousal seem to be less affected by mono-method bias, whereas more complex perceptual constructs such as engagement or cognitive load have higher within method correlations. There are two complementary explanations for the within method correlations: (a) a combination between complexity of trait and method and (b) method effects that are congeneric.
ISSN:0747-5632
1873-7692
DOI:10.1016/j.chb.2012.12.009