Verification of predicted robustness and accuracy of multivariate analysis

• Published in: NeuroImage (Orlando, Fla.) Fla.), 2011-06, Vol.56 (3), p.1382-1385
• Main Authors: Markiewicz, P.J., Matthews, J.C., Declerck, J., Herholz, K.
• Format: Article
• Language: English
• Subjects: Accuracy; Aged; Alzheimer Disease - diagnostic imaging; Alzheimer's disease; Bootstrap; Databases, Factual; Discriminant Analysis; Female; Fluorodeoxyglucose F18; Humans; Image Processing, Computer-Assisted - methods; Male; Medical imaging; Methods; Multivariate Analysis; Multivariate image analysis; PET; Pharmaceutical industry; Positron-Emission Tomography - methods; Principal Component Analysis; Radiopharmaceuticals; Reproducibility of Results; Resampling; Robustness; Sample size; Scanners; Statistical analysis; Studies
• ISSN: 1053-8119; 1095-9572
• DOI: 10.1016/j.neuroimage.2011.02.036

The assessment of accuracy and robustness of multivariate analysis of FDG-PET brain images as presented in [Markiewicz, P.J., Matthews, J.C., Declerck, J., Herholz, K., 2009. Robustness of multivariate image analysis assessed by resampling techniques and applied to FDG-PET scans of patients with Alzheimer's disease. Neuroimage 46, 472–485.] using a homogeneous sample (from one centre) of small size is here verified using a heterogeneous sample (from multiple centres) of much larger size. Originally the analysis, which included principal component analysis (PCA) and Fisher discriminant analysis (FDA), was established using a sample of 42 subjects (19 Normal Controls (NCs) and 23 Alzheimer's disease (AD) patients) and here the analysis is verified using an independent sample of 166 subjects (86 NCs and 80 ADs) obtained from the ADNI database. It is shown that bootstrap resampling combined with the metric of the largest principal angle between PCA subspaces as well as the deliberate clinical misdiagnosis simulation can predict robustness of the multivariate analysis when used with new datasets. Cross-validation (CV) and the .632 bootstrap overestimated the predictive accuracy encouraging less robust solutions. Also, it is shown that the type of PET scanner and image reconstruction method has an impact on such analysis and affects the accuracy of the verification sample. ► Verification of model selection methods for a small sample (42). ► Use of a much larger sample (166) from ADNI for the verification of the PCA/FDA model. ► The .632 bootstrap or CV estimators can fail encouraging unrobust solutions. ► Bootstrap with the proposed metric can well predict robustness of chosen models.► Scanner types and image reconstruction methods had an effect on the verification.