Predicting Release Time based on Software Reliability Model

Predicting Release Time based on Software Reliability Model

Hironori Washizaki, Kiyoshi Honda, Yoshiaki Fukazawa

Waseda University

Twitter: @Hiro_Washi

[email protected]

http://www.washi.cs.waseda.ac.jp/

Agile 2015 Research Track, August 6th, 2015

When to release? • Predicting completion based on cumulative

flow diagram [Power, Agile’14]

2

Ken Power, "Metrics for Understanding Flow,“ Proceedings of the Agile 2014 Conference, 2014.

O blog da ASPERCOM Treinamentos, O Cumulative Flow Diagram,

http://blog.aspercom.com.br/2012/04/03/cumulative-flow-diagram/

T1 T2

Done

WIP

Backlog

• It could have a broad range of completion time.

• Ready to release in terms of reliability?

Software reliability model (SRM)

3

#Issues

Actual

Predicted

Days K. Honda, et al., Predicting Time Range Based on Generalized Software Reliability Model , APSEC’14

Logistic

Gompertz

Non-homogeneous Poisson

process(NHPP)

Our challenges in SRM

1. Predicting past or future?

2. Prediction with respect to each release, iteration, or testing level?

3. Uncertainty?

4

Our challenges in SRM

1. Predicting past or future?

2. Prediction with respect to each release, iteration, or testing level?

3. Uncertainty?

5

Predicting, what?

Future!

6

Mbz1, The fossils from Cretaceous age found in Lebanon.jpg CC BY-SA 3.0

https://en.wikipedia.org/wiki/Fossil#/media/File:The_fossils_from_Cretaceou

s_age_found_in_Lebanon.jpg

Past?

Future NEXT EXIT

7

#Issues

Actual

Predicted

Day K. Honda, et al., Predicting Time Range Based on Generalized Software Reliability Model , APSEC’14

Industrial case

SRM for runtime future prediction

8

0

5

10

15

20

25

0 0.2 0.4 0.6 0.8 1

#Issues #Total issues predicted

at each time point

75 75 300

SRM for runtime future prediction

9

0

5

10

15

20

25

0 0.2 0.4 0.6 0.8 1

#Total issues predicted

at each time point

75 75 120 120

#Issues

200

10

#Issues #Total issues predicted

at each time point

75 120

#Issues

Something happened!

Predication became stable.

SRM as actionable metric!

Our challenges in SRM

1. Predicting past or future?

2. Prediction with respect to each release, iteration, or testing level?

3. Uncertainty?

11

Prediction with respect to release: a OSS case

“Foundation” http://foundation.zurb.com/

12

438 500

Predicted release date: 498

Our challenges in SRM

1. Predicting past or future?

2. Prediction with respect to each release, iteration, or testing level?

3. Uncertainty?

14

Uncertainty

15 15

Prediction with uncertainty

0

10

20

30

40

50

60

70

80

90

0 2 4 6 8 10 12 14

#Is

su

es

Time

ActualData

Our model

16

Prediction with uncertainty

0

10

20

30

40

50

60

70

80

90

0 2 4 6 8 10 12 14

#Is

su

es

Time

ActualData

Our model

-

+

17

Lower (worst case)

Upper (best case)

T1 T2

Uncertainty patterns and prediction

18

0 0.5 1

Constant Increase Decrease

Conclusion and future work

• Our contributions: using SRM – Predicting future

– Prediction with respect to release and testing level

– Uncertainty patterns

– Tool available as Jenkins plug-in

19

https://jenkins-ci.org/

• Future work – How to specify appropriate uncertainty pattern?

– Prediction with respect to release, testing level or iteration?

– How work well for Agile developments?

– Combination of feature completion (Cumulative Flow Diagram) and issues (Software Reliability Model)?

Thank you very much! Questions?

• [email protected]

• @Hiro_Washi

20