Exact Mean Absolute Error of Baseline Predictor, MARP0

Shepperd and MacDonell “Evaluating prediction systems in software project estimation”. Information and Software Technology 54 (8), 820–827, 2012, proposed an improved measure of the effectiveness of predictors based on comparing them with random guessing. They suggest estimating the performance of r...

Full description

Saved in:
Bibliographic Details
Published in:Information and software technology 2016-05, Vol.73, p.16-18
Main Authors: Langdon, William B., Dolado, Javier, Sarro, Federica, Harman, Mark
Format: Article
Language:English
Subjects:
Comparative analysis
Computer programs
Effectiveness studies
Empirical validation
Errors
Estimates
Estimating
Estimating techniques
Estimation bias
Mathematical analysis
Monte Carlo methods
Monte Carlo simulation
Prediction systems
Predictions
Randomisation techniques
Search based software engineering
Software
Software engineering
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Shepperd and MacDonell “Evaluating prediction systems in software project estimation”. Information and Software Technology 54 (8), 820–827, 2012, proposed an improved measure of the effectiveness of predictors based on comparing them with random guessing. They suggest estimating the performance of random guessing using a Monte Carlo scheme which unfortunately excludes some correct guesses. This biases their MARP0 to be slightly too big, which in turn causes their standardised accuracy measure SA to over estimate slightly. In commonly used software engineering datasets it is practical to calculate an unbiased MARP0 exactly.
ISSN:0950-5849
1873-6025
DOI:10.1016/j.infsof.2016.01.003