Model validation software for classification models using repeated partitioning: MVREP

The process of assessing the prediction ability of a computational model is called model validation. For models predicting a categorical response, the prediction ability is usually quantified by prediction measures such as sensitivity, specificity, and accuracy. This paper presents a software Model...

Full description

Saved in:
Bibliographic Details
Published in:Computer methods and programs in biomedicine 2003-09, Vol.72 (1), p.81-87
Main Authors: Li, Wei, Arena, Vincent C., Sussman, Nancy B., Mazumdar, Sati
Format: Article
Language:English
Subjects:
Animals
Biological and medical sciences
Classification
Classification models
Computer Simulation
Humans
Logistic Models
Medical sciences
Mice
Model validation
Prediction error
Repartitioning
Risk Assessment
Software
Toxicology - statistics & numerical data
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The process of assessing the prediction ability of a computational model is called model validation. For models predicting a categorical response, the prediction ability is usually quantified by prediction measures such as sensitivity, specificity, and accuracy. This paper presents a software Model Validation using Repeated Partitioning (MVREP) that implements a computer-intensive, nonparametric approach to model validation, which we call the re-partitioning method. MVREP, developed using the SAS Macro language, repeats the process of randomly partitioning a dataset and subsequently performing standard model validation procedures, such as cross-validation, a large number of times and generates the empirical sampling distributions of prediction measures. The means of the sampling distributions serve as the point estimates of prediction measures of the model. The variances of the sampling distributions provide a direct assessment of variability for the point estimates of prediction measures. An example is presented using a mouse developmental toxicity chemical dataset to illustrate how the software can be used for the assessment of structure-activity relationships models.
ISSN:0169-2607
1872-7565
DOI:10.1016/S0169-2607(02)00119-0