Evaluating the Effects of Rewards and Schedule Length on Response Rates to Ecological Momentary Assessment Surveys: Randomized Controlled Trials

• Published in: Journal of medical Internet research 2023-10, Vol.25 (1), p.e45764-e45764
• Main Authors: Edney, Sarah, Goh, Claire Marie, Chua, Xin Hui, Low, Alicia, Chia, Janelle, S Koek, Daphne, Cheong, Karen, van Dam, Rob, Tan, Chuen Seng, Müller-Riemenschneider, Falk
• Format: Article
• Language: English
• Subjects: Analysis of covariance; Biometrics; Changes; Clinical trials; Cohort analysis; College students; Ecological momentary assessment; Efficacy; Email; Exercise; Health behavior; Longitudinal studies; Missing data; Original Paper; Polls & surveys; Questionnaires; Response rates; Rewards; Smartphones; Text messaging
• ISSN: 1438-8871; 1439-4456; 1438-8871
• DOI: 10.2196/45764

Ecological momentary assessments (EMAs) are short, repeated surveys designed to collect information on experiences in real-time, real-life contexts. Embedding periodic bursts of EMAs within cohort studies enables the study of experiences on multiple timescales and could greatly enhance the accuracy of self-reported information. However, the burden on participants may be high and should be minimized to optimize EMA response rates. We aimed to evaluate the effects of study design features on EMA response rates. Embedded within an ongoing cohort study (Health@NUS), 3 bursts of EMAs were implemented over a 7-month period (April to October 2021). The response rate (percentage of completed EMA surveys from all sent EMA surveys; 30-42 individual EMA surveys sent/burst) for each burst was examined. Following a low response rate in burst 1, changes were made to the subsequent implementation strategy (SMS text message announcements instead of emails). In addition, 2 consecutive randomized controlled trials were conducted to evaluate the efficacy of 4 different reward structures (with fixed and bonus components) and 2 different schedule lengths (7 or 14 d) on changes to the EMA response rate. Analyses were conducted from 2021 to 2022 using ANOVA and analysis of covariance to examine group differences and mixed models to assess changes across all 3 bursts. Small changes to the implementation strategy (SMS text messages instead of emails) may have contributed to increasing the response rate over time. Changing the available rewards did not lead to a significant difference in the response rate, whereas changing the schedule length did lead to a significant difference in the response rate. Our study provides novel insights on how to implement EMA surveys in ongoing cohort studies. This knowledge is essential for conducting high-quality studies using EMA surveys.