Using Correlation and Accuracy for Identifying Good Estimators http:// nas.cl.uh.edu/boetticher/publications.htm l The 4 th International Predictor Models in Software Engineering (PROMISE) Workshop Gary D. Boetticher Nazim Lokhandwala Univ. of Houston - Clear Lake, Houston, TX, USA [email protected][email protected]63 62 61
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Using Correlation and Accuracy for Identifying Good Estimators
http://nas.cl.uh.edu/boetticher/publications.html The 4th International Predictor Models in Software Engineering (PROMISE) Workshop
Gary D. Boetticher Nazim Lokhandwala Univ. of Houston - Clear Lake, Houston, TX, USA
http://nas.cl.uh.edu/boetticher/publications.html The 4th International Predictor Models in Software Engineering (PROMISE) Workshop
2-Class Problem: 10 Best (A), 10 Worst (F)
1000 Trials,Accuracy = 41.6%
Attribute Reductionusing WRAPPER
1000 Trials,Accuracy = 78.6%
Results: Scale Factor Only
http://nas.cl.uh.edu/boetticher/publications.html The 4th International Predictor Models in Software Engineering (PROMISE) Workshop
1000 Trials,Accuracy = 65.0%
Attribute Reductionusing WRAPPER
1000 Trials,Accuracy = 78.2%
2-Class Problem: 10 Best (A), 10 Worst (F)
Results: Correlation & Scale Factor
http://nas.cl.uh.edu/boetticher/publications.html The 4th International Predictor Models in Software Engineering (PROMISE) Workshop
1000 Trials,Accuracy = 82.2%
Attribute Reductionusing WRAPPER
1000 Trials,Accuracy = 93.3%
2-Class Problem: 10 Best (A), 10 Worst (F)
Discussion - 1
http://nas.cl.uh.edu/boetticher/publications.html The 4th International Predictor Models in Software Engineering (PROMISE) Workshop
Best Estimators
Poorest Estimators
Average Correlation 0.4173 0.3686
Average Scale Factor 2.6198 2.7419
How well does the decision tree from the third experiment apply to all the respondents minus outliers?
Discussion - 2
http://nas.cl.uh.edu/boetticher/publications.html The 4th International Predictor Models in Software Engineering (PROMISE) Workshop
Scope of effort
Amortization of effort
Reuse can skew estimates (esp. Design for Reuse)
Respondent’s estimates = Boetticher’s estimates
Challenges in component effort estimation
Conclusions
Good accuracy rates,
especially after attribute reduction
Correlation + Scale Factor Intuitive Model
Bridges expert and model groups
http://nas.cl.uh.edu/boetticher/publications.html The 4th International Predictor Models in Software Engineering (PROMISE) Workshop
http://nas.cl.uh.edu/boetticher/publications.html
Thank You !
The 4th International Predictor Models in Software Engineering (PROMISE) Workshop
References
1) Jorgensen, M., “A review of studies on Expert Estimation of Software Development Effort,” Journal of Systems and Software, 2004.
2) Jørgensen, Shepperd, A Systematic Review of Software Development Cost Estimation Studies, IEEE Transactions on Software Engineering, 33, 1, January, 2007, Pp. 33-53.
The 4th International Predictor Models in Software Engineering (PROMISE) Workshop