Exact Mean Absolute Error of Baseline Predictor, MARP0

Shepperd and MacDonell “Evaluating prediction systems in software project estimation”. Information and Software Technology 54 (8), 820–827, 2012, proposed an improved measure of the effectiveness of predictors based on comparing them with random guessing. They suggest estimating the performance of r...

Full description

Saved in:
Bibliographic Details
Published in:Information and software technology 2016-05, Vol.73, p.16-18
Main Authors: Langdon, William B., Dolado, Javier, Sarro, Federica, Harman, Mark
Format: Article
Language:English
Subjects:
Comparative analysis
Computer programs
Effectiveness studies
Empirical validation
Errors
Estimates
Estimating
Estimating techniques
Estimation bias
Mathematical analysis
Monte Carlo methods
Monte Carlo simulation
Prediction systems
Predictions
Randomisation techniques
Search based software engineering
Software
Software engineering
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c348t-3ecf054e0c3a9916ac1e96c33c5cbc7f5b9f0a24a9741e83b32920d153572fa93
cites cdi_FETCH-LOGICAL-c348t-3ecf054e0c3a9916ac1e96c33c5cbc7f5b9f0a24a9741e83b32920d153572fa93
container_end_page 18
container_issue
container_start_page 16
container_title Information and software technology
container_volume 73
creator Langdon, William B.
Dolado, Javier
Sarro, Federica
Harman, Mark
description Shepperd and MacDonell “Evaluating prediction systems in software project estimation”. Information and Software Technology 54 (8), 820–827, 2012, proposed an improved measure of the effectiveness of predictors based on comparing them with random guessing. They suggest estimating the performance of random guessing using a Monte Carlo scheme which unfortunately excludes some correct guesses. This biases their MARP0 to be slightly too big, which in turn causes their standardised accuracy measure SA to over estimate slightly. In commonly used software engineering datasets it is practical to calculate an unbiased MARP0 exactly.
doi_str_mv 10.1016/j.infsof.2016.01.003
format article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_1816091064</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S0950584916000057</els_id><sourcerecordid>1816091064</sourcerecordid><originalsourceid>FETCH-LOGICAL-c348t-3ecf054e0c3a9916ac1e96c33c5cbc7f5b9f0a24a9741e83b32920d153572fa93</originalsourceid><addsrcrecordid>eNp9kE1LxDAQhoMouK7-Aw8FLx5snTRN21yEdVk_YBcX0XNIsxNI6TZr0hX992apJw-ehoHnfZl5CLmkkFGg5W2b2d4EZ7I8bhnQDIAdkQmtK5aWkPNjMgHBIeV1IU7JWQgtAK2AwYSUiy-lh2SFqk9mTXDdfsBk4b3ziTPJvQrY2R6TtceN1YPzN8lq9rqGc3JiVBfw4ndOyfvD4m3-lC5fHp_ns2WqWVEPKUNtgBcImikhaKk0RVFqxjTXja4Mb4QBlRdKVAXFmjUsFzlsKGe8yo0SbEqux96ddx97DIPc2qCx61SPbh8krWkJgkJZRPTqD9q6ve_jdZJWVZUXdV3mkSpGSnsXgkcjd95ulf-WFORBpmzlKFMeZEqgMsqMsbsxhvHZT4teBm2x19GKRz3IjbP_F_wATUB8GQ</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>1777248862</pqid></control><display><type>article</type><title>Exact Mean Absolute Error of Baseline Predictor, MARP0</title><source>ScienceDirect Freedom Collection 2022-2024</source><creator>Langdon, William B. ; Dolado, Javier ; Sarro, Federica ; Harman, Mark</creator><creatorcontrib>Langdon, William B. ; Dolado, Javier ; Sarro, Federica ; Harman, Mark</creatorcontrib><description>Shepperd and MacDonell “Evaluating prediction systems in software project estimation”. Information and Software Technology 54 (8), 820–827, 2012, proposed an improved measure of the effectiveness of predictors based on comparing them with random guessing. They suggest estimating the performance of random guessing using a Monte Carlo scheme which unfortunately excludes some correct guesses. This biases their MARP0 to be slightly too big, which in turn causes their standardised accuracy measure SA to over estimate slightly. In commonly used software engineering datasets it is practical to calculate an unbiased MARP0 exactly.</description><identifier>ISSN: 0950-5849</identifier><identifier>EISSN: 1873-6025</identifier><identifier>DOI: 10.1016/j.infsof.2016.01.003</identifier><language>eng</language><publisher>Amsterdam: Elsevier B.V</publisher><subject>Comparative analysis ; Computer programs ; Effectiveness studies ; Empirical validation ; Errors ; Estimates ; Estimating ; Estimating techniques ; Estimation bias ; Mathematical analysis ; Monte Carlo methods ; Monte Carlo simulation ; Prediction systems ; Predictions ; Randomisation techniques ; Search based software engineering ; Software ; Software engineering</subject><ispartof>Information and software technology, 2016-05, Vol.73, p.16-18</ispartof><rights>2016 Elsevier B.V.</rights><rights>Copyright Elsevier Science Ltd. May 2016</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c348t-3ecf054e0c3a9916ac1e96c33c5cbc7f5b9f0a24a9741e83b32920d153572fa93</citedby><cites>FETCH-LOGICAL-c348t-3ecf054e0c3a9916ac1e96c33c5cbc7f5b9f0a24a9741e83b32920d153572fa93</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,776,780,27901,27902</link.rule.ids></links><search><creatorcontrib>Langdon, William B.</creatorcontrib><creatorcontrib>Dolado, Javier</creatorcontrib><creatorcontrib>Sarro, Federica</creatorcontrib><creatorcontrib>Harman, Mark</creatorcontrib><title>Exact Mean Absolute Error of Baseline Predictor, MARP0</title><title>Information and software technology</title><description>Shepperd and MacDonell “Evaluating prediction systems in software project estimation”. Information and Software Technology 54 (8), 820–827, 2012, proposed an improved measure of the effectiveness of predictors based on comparing them with random guessing. They suggest estimating the performance of random guessing using a Monte Carlo scheme which unfortunately excludes some correct guesses. This biases their MARP0 to be slightly too big, which in turn causes their standardised accuracy measure SA to over estimate slightly. In commonly used software engineering datasets it is practical to calculate an unbiased MARP0 exactly.</description><subject>Comparative analysis</subject><subject>Computer programs</subject><subject>Effectiveness studies</subject><subject>Empirical validation</subject><subject>Errors</subject><subject>Estimates</subject><subject>Estimating</subject><subject>Estimating techniques</subject><subject>Estimation bias</subject><subject>Mathematical analysis</subject><subject>Monte Carlo methods</subject><subject>Monte Carlo simulation</subject><subject>Prediction systems</subject><subject>Predictions</subject><subject>Randomisation techniques</subject><subject>Search based software engineering</subject><subject>Software</subject><subject>Software engineering</subject><issn>0950-5849</issn><issn>1873-6025</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2016</creationdate><recordtype>article</recordtype><recordid>eNp9kE1LxDAQhoMouK7-Aw8FLx5snTRN21yEdVk_YBcX0XNIsxNI6TZr0hX992apJw-ehoHnfZl5CLmkkFGg5W2b2d4EZ7I8bhnQDIAdkQmtK5aWkPNjMgHBIeV1IU7JWQgtAK2AwYSUiy-lh2SFqk9mTXDdfsBk4b3ziTPJvQrY2R6TtceN1YPzN8lq9rqGc3JiVBfw4ndOyfvD4m3-lC5fHp_ns2WqWVEPKUNtgBcImikhaKk0RVFqxjTXja4Mb4QBlRdKVAXFmjUsFzlsKGe8yo0SbEqux96ddx97DIPc2qCx61SPbh8krWkJgkJZRPTqD9q6ve_jdZJWVZUXdV3mkSpGSnsXgkcjd95ulf-WFORBpmzlKFMeZEqgMsqMsbsxhvHZT4teBm2x19GKRz3IjbP_F_wATUB8GQ</recordid><startdate>201605</startdate><enddate>201605</enddate><creator>Langdon, William B.</creator><creator>Dolado, Javier</creator><creator>Sarro, Federica</creator><creator>Harman, Mark</creator><general>Elsevier B.V</general><general>Elsevier Science Ltd</general><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope></search><sort><creationdate>201605</creationdate><title>Exact Mean Absolute Error of Baseline Predictor, MARP0</title><author>Langdon, William B. ; Dolado, Javier ; Sarro, Federica ; Harman, Mark</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c348t-3ecf054e0c3a9916ac1e96c33c5cbc7f5b9f0a24a9741e83b32920d153572fa93</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2016</creationdate><topic>Comparative analysis</topic><topic>Computer programs</topic><topic>Effectiveness studies</topic><topic>Empirical validation</topic><topic>Errors</topic><topic>Estimates</topic><topic>Estimating</topic><topic>Estimating techniques</topic><topic>Estimation bias</topic><topic>Mathematical analysis</topic><topic>Monte Carlo methods</topic><topic>Monte Carlo simulation</topic><topic>Prediction systems</topic><topic>Predictions</topic><topic>Randomisation techniques</topic><topic>Search based software engineering</topic><topic>Software</topic><topic>Software engineering</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Langdon, William B.</creatorcontrib><creatorcontrib>Dolado, Javier</creatorcontrib><creatorcontrib>Sarro, Federica</creatorcontrib><creatorcontrib>Harman, Mark</creatorcontrib><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>Information and software technology</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Langdon, William B.</au><au>Dolado, Javier</au><au>Sarro, Federica</au><au>Harman, Mark</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Exact Mean Absolute Error of Baseline Predictor, MARP0</atitle><jtitle>Information and software technology</jtitle><date>2016-05</date><risdate>2016</risdate><volume>73</volume><spage>16</spage><epage>18</epage><pages>16-18</pages><issn>0950-5849</issn><eissn>1873-6025</eissn><abstract>Shepperd and MacDonell “Evaluating prediction systems in software project estimation”. Information and Software Technology 54 (8), 820–827, 2012, proposed an improved measure of the effectiveness of predictors based on comparing them with random guessing. They suggest estimating the performance of random guessing using a Monte Carlo scheme which unfortunately excludes some correct guesses. This biases their MARP0 to be slightly too big, which in turn causes their standardised accuracy measure SA to over estimate slightly. In commonly used software engineering datasets it is practical to calculate an unbiased MARP0 exactly.</abstract><cop>Amsterdam</cop><pub>Elsevier B.V</pub><doi>10.1016/j.infsof.2016.01.003</doi><tpages>3</tpages></addata></record>
fulltext fulltext
identifier ISSN: 0950-5849
ispartof Information and software technology, 2016-05, Vol.73, p.16-18
issn 0950-5849
1873-6025
language eng
recordid cdi_proquest_miscellaneous_1816091064
source ScienceDirect Freedom Collection 2022-2024
subjects Comparative analysis
Computer programs
Effectiveness studies
Empirical validation
Errors
Estimates
Estimating
Estimating techniques
Estimation bias
Mathematical analysis
Monte Carlo methods
Monte Carlo simulation
Prediction systems
Predictions
Randomisation techniques
Search based software engineering
Software
Software engineering
title Exact Mean Absolute Error of Baseline Predictor, MARP0
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-03T09%3A29%3A47IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Exact%20Mean%20Absolute%20Error%20of%20Baseline%20Predictor,%20MARP0&rft.jtitle=Information%20and%20software%20technology&rft.au=Langdon,%20William%20B.&rft.date=2016-05&rft.volume=73&rft.spage=16&rft.epage=18&rft.pages=16-18&rft.issn=0950-5849&rft.eissn=1873-6025&rft_id=info:doi/10.1016/j.infsof.2016.01.003&rft_dat=%3Cproquest_cross%3E1816091064%3C/proquest_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c348t-3ecf054e0c3a9916ac1e96c33c5cbc7f5b9f0a24a9741e83b32920d153572fa93%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=1777248862&rft_id=info:pmid/&rfr_iscdi=true