Behavior, sensitivity, and power of activation likelihood estimation characterized by massive empirical simulation

• Published in: NeuroImage (Orlando, Fla.) Fla.), 2016-08, Vol.137, p.70-85
• Main Authors: Eickhoff, Simon B., Nichols, Thomas E., Laird, Angela R., Hoffstaedter, Felix, Amunts, Katrin, Fox, Peter T., Bzdok, Danilo, Eickhoff, Claudia R.
• Format: Article
• Language: English
• Subjects: Brain; Brain - physiology; Cognition & reasoning; Data Interpretation, Statistical; Error correction & detection; Experiments; Humans; Image Interpretation, Computer-Assisted - methods; Image processing; Inference; Language; Likelihood Functions; Medical imaging; Memory; Meta-analysis; Meta-Analysis as Topic; Neuroimaging; Neuroimaging - methods; NMR; Nuclear magnetic resonance; Orthography; Phonology; Quantitative analysis; Reading; Reproducibility of Results; Schizophrenia; Semantics; Sensitivity and Specificity; Speech; Statistics; Systematic review
• ISSN: 1053-8119; 1095-9572
• DOI: 10.1016/j.neuroimage.2016.04.072

Given the increasing number of neuroimaging publications, the automated knowledge extraction on brain-behavior associations by quantitative meta-analyses has become a highly important and rapidly growing field of research. Among several methods to perform coordinate-based neuroimaging meta-analyses, Activation Likelihood Estimation (ALE) has been widely adopted. In this paper, we addressed two pressing questions related to ALE meta-analysis: i) Which thresholding method is most appropriate to perform statistical inference? ii) Which sample size, i.e., number of experiments, is needed to perform robust meta-analyses? We provided quantitative answers to these questions by simulating more than 120,000 meta-analysis datasets using empirical parameters (i.e., number of subjects, number of reported foci, distribution of activation foci) derived from the BrainMap database. This allowed to characterize the behavior of ALE analyses, to derive first power estimates for neuroimaging meta-analyses, and to thus formulate recommendations for future ALE studies. We could show as a first consequence that cluster-level family-wise error (FWE) correction represents the most appropriate method for statistical inference, while voxel-level FWE correction is valid but more conservative. In contrast, uncorrected inference and false-discovery rate correction should be avoided. As a second consequence, researchers should aim to include at least 20 experiments into an ALE meta-analysis to achieve sufficient power for moderate effects. We would like to note, though, that these calculations and recommendations are specific to ALE and may not be extrapolated to other approaches for (neuroimaging) meta-analysis.