Using machine learning to analyze longitudinal data: A tutorial guide and best‐practice recommendations for social science researchers

This article introduces the research community to the power of machine learning over traditional approaches when analyzing longitudinal data. Although traditional approaches work well with small to medium datasets, machine learning models are more appropriate as the available data becomes larger and...

Full description

Saved in:
Bibliographic Details
Published in:Applied psychology 2023-07, Vol.72 (3), p.1339-1364
Main Authors: Sheetal, Abhishek, Jiang, Zhou, Di Milia, Lee
Format: Article
Language:English
Subjects:
Big Five personality
Cognitive style
longitudinal data
Machine learning
Neuroticism
Partial information
Polls & surveys
Social sciences
Solomonoff induction
XGBoost
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This article introduces the research community to the power of machine learning over traditional approaches when analyzing longitudinal data. Although traditional approaches work well with small to medium datasets, machine learning models are more appropriate as the available data becomes larger and more complex. Additionally, machine learning methods are ideal for analyzing longitudinal data because they do not make any assumptions about the distribution of the dependent and independent variables or the homogeneity of the underlying population. They can also analyze cases with partial information. In this article, we use the Household, Income, and Labour Dynamics in Australia (HILDA) survey to illustrate the benefits of machine learning. Using a machine learning algorithm, we analyze the relationship between job‐related variables and neuroticism across 13 years of the HILDA survey. We suggest that the results produced by machine learning can be used to generate generalizable rules from the data to augment our theoretical understanding of the domain. With a technical guide, this article offers critical information and best‐practice recommendations that can assist social science researchers in conducting machine learning analysis with longitudinal data.
ISSN:0269-994X
1464-0597
DOI:10.1111/apps.12435