The role of outcome feedback in improving the uncertainty assessment of software development effort estimates

• Published in: ACM transactions on software engineering and methodology 2008-08, Vol.17 (4), p.1-35
• Main Authors: Gruschke, Tanja M., Jørgensen, Magne
• Format: Article
• Language: English
• ISSN: 1049-331X; 1557-7392
• DOI: 10.1145/13487689.13487693

Previous studies report that software developers are over-confident in the accuracy of their effort estimates. Aim: This study investigates the role of outcome feedback, that is, feedback about the discrepancy between the estimated and the actual effort, in improving the uncertainty assessments. Method: We conducted two in-depth empirical studies on uncertainty assessment learning. Study 1 included five student developers and Study 2, 10 software professionals. In each study the developers repeatedly assessed the uncertainty of their effort estimates of a programming task, solved the task, and received estimation accuracy outcome feedback. Results: We found that most, but not all, developers were initially over-confident in the accuracy of their effort estimates and remained over-confident in spite of repeated and timely outcome feedback. One important, but not sufficient, condition for improvement based on outcome feedback seems to be the use of explicitly formulated, instead of purely intuition-based, uncertainty assessment strategies.