Assessing the statistical validity of proteomics based biomarkers

• Published in: Analytica chimica acta 2007-06, Vol.592 (2), p.210-217
• Main Authors: Smit, Suzanne, van Breemen, Mariëlle J., Hoefsloot, Huub C.J., Smilde, Age K., Aerts, Johannes M.F.G., de Koster, Chris G.
• Format: Article
• Language: English
• Subjects: Adolescent; Adult; Aged; Analytical chemistry; Biomarker discovery; Biomarkers - blood; Biomarkers - chemistry; Chemistry; Classification; Curse of dimensionality; Double cross-validation; Exact sciences and technology; Female; Gaucher disease; Humans; Male; Mass Spectrometry; Middle Aged; Permutation test; Principal component discriminant analysis; Proteomics - classification; Proteomics - methods; Proteomics - standards; Rank products; Reproducibility of Results; Spectrometric and optical methods; Statistical validation; Statistics as Topic; Surface enhanced laser desorption ionization time-of-flight mass spectrometry
• ISSN: 0003-2670; 1873-4324
• DOI: 10.1016/j.aca.2007.04.043

A strategy is presented for the statistical validation of discrimination models in proteomics studies. Several existing tools are combined to form a solid statistical basis for biomarker discovery that should precede a biochemical validation of any biomarker. These tools consist of permutation tests, single and double cross-validation. The cross-validation steps can simply be combined with a new variable selection method, called rank products. The strategy is especially suited for the low-samples-to-variables-ratio (undersampling) case, as is often encountered in proteomics and metabolomics studies. As a classification method, principal component discriminant analysis is used; however, the methodology can be used with any classifier. A dataset containing serum samples from Gaucher patients and healthy controls serves as a test case. Double cross-validation shows that the sensitivity of the model is 89% and the specificity 90%. Potential putative biomarkers are identified using the novel variable selection method. Results from permutation tests support the choice of double cross-validation as the tool for determining error rates when the modelling procedure involves a tuneable parameter. This shows that even cross-validation does not guarantee unbiased results. The validation of discrimination models with a combination of permutation tests and double cross-validation helps to avoid erroneous results which may result from the undersampling.