Methodical advances in reproducibility research: A proof of concept qualitative comparative analysis of reproducing animal data in humans

• Published in: Journal of neuroscience methods 2023-09, Vol.397, p.109931-109931, Article 109931
• Main Authors: Leenaars, Cathalijn H.C., Teerenstra, Steven, Meijboom, Franck L.B., Bleich, André
• Format: Article
• Language: English
• Subjects: Extrapolation; Modelling; Prediction; Qualitative comparative analysis; Regression analysis; Reproducibility; Scoping review; Validity
• ISSN: 0165-0270; 1872-678X
• DOI: 10.1016/j.jneumeth.2023.109931

While the term reproducibility crisis mainly reflects reproducibility of experiments between laboratories, reproducibility between species also remains problematic. We previously summarised the published reproducibility between animal and human studies; i.e. the translational success rates, which varied from 0% to 100%. Based on analyses of individual factors, we could not predict reproducibility. Several potential analyses can assess effect of combinations of predictors on an outcome. Regression analysis (RGA) is common, but not ideal to analyse multiple interactions and specific configurations (≈ combinations) of variables, which could be highly relevant to reproducibility. Qualitative comparative analysis (QCA) is based on set theory and Boolean algebra, and was successfully used in other fields. We reanalysed the data from our preceding review with QCA. This QCA resulted in the following preliminary formula for successful translation: ∼Old*∼Intervention*∼Large*MultSpec*Quantitative Which means that within the analysed dataset, the combination of relative recency (∼ means not; >1999), analyses at event or study level (not at intervention level), n 85%). Other combinations of factors showed less consistent or negative results. An RGA on the same data did not identify any of the included variables as significant contributors. While these data were not collected with the QCA in mind, they illustrate that the approach is viable and relevant for this research field. The QCA seems a highly promising approach to furthering our knowledge on between-species reproducibility. •Analyses to predict reproducibility between species are challenging•Reproducibility between species varies from 0% to 100%.•QCA is a promising approach to increase knowledge on between-species reproducibility•QCA is more sensitive than regression for analysing combinations of factors•Small quantitative analyses of multiple species may improve reproducibility