Introducing DigiCAT: A digital tool to promote the principled use of counterfactual analysis for identifying potential active ingredients in mental health

• Published in: Wellcome open research 2024, Vol.9, p.376
• Main Authors: Murray, Aja Louise, Wright, Helen, Casey, Hannah, Yang, Yi, Zhu, Xinxin, Obsuth, Ingrid, Allitt, Marie, Mirman, Dan, Errington, Patrick, King, Josiah
• Format: Article
• Language: English
• ISSN: 2398-502X; 2398-502X
• DOI: 10.12688/wellcomeopenres.21105.1

Background Given the challenges and resources involved in mental health intervention development and evaluation, it is valuable to obtain early evidence on which intervention targets represent the most promising investments. Observational datasets provide a rich resource for exploring these types of questions; however, the lack of randomisation to treatments in these data means they are vulnerable to confounding issues. Counterfactual analysis refers to a family of techniques within the potential outcomes framework that can help address confounding. In doing so, they can help differentiate potential intervention targets that may reflect genuine active ingredients in mental health from those that are only associated with mental health outcomes due to their common dependence on ‘third variables’. However, counterfactual analysis is rarely used for this purpose and where it is used in health research it is often implemented in a suboptimal fashion. One key reason may be a lack of accessible tutorials and software that embeds best practices. Methods To help promote the principled use of counterfactual analysis we developed DigiCAT. DigiCAT is an open digital tool built in R and Shiny that implements a range of counterfactual analysis methods. It is accompanied by accessible tutorials. The tool has been designed to handle real data, with capabilities for missing data, non-binary treatment effects, and complex survey designs. Results The current article describes the development of DigiCAT, drawing on user and lived experience expert input and provides an overview of its features and examples of its uses. Conclusions Counterfactual analysis could help prioritise intervention targets by establishing which ones remain associated with mental health outcomes after accounting for potential confounding. Accessible digital tools supported by clear guidance may help promote the uptake and principled use of these techniques.