2180_Nandyal

© SITARA Technologies

For Use and Distribution by the SEI in the proceedings of the 24th SEPG Conference 2012 – Albuquerque 1111

SITARA Process JewelBoxTM

© SITARA Technologies Pvt. Ltd.


How to Accelerate High-Maturity Implementation of CMMI using Six Sigma

Raghav S. NandyalChief Executive Officer

SITARA Technologies Pvt. Ltd.

[email protected]

Work-product of the 2011 High-maturity Partner Advisory Board of the SEI.

I thank: Bradley Bittorf, Eberhard Huebner and Kaliappan Marrapa for their

feedback.






LEARNER OBJECTIVES

Why should you consider Six Sigma as a problem solving methodology to accelerate High Maturity process institutionalization?

How can Six Sigma help in accelerating quantitative benefits while assuring a statistical significance?

How can a simple 4-step process help to demystify Six Sigma application in the context of process improvement?






Some Perspectives …

Why a blind application of Six Sigma might be wrong? (1 of 5)

Noteworthy distinctions while considering the

application of Six Sigma to a software process:

Six Sigma originated in manufacturing and extended itself to other transactional services

Most Six Sigma analysis techniques, tools and

templates are well suited to the needs of a

manufacturing or a transactional service

Well known optimization procedures (e.g.,

DOE), approaches to assess process stability

(e.g., Western Electric rules) and capability

analysis (long-term and short-term capability)

don’t extend naturally to a software process

Quality assurance and not quality control – is

more relevant to a software development

process based on the CMMI






Noteworthy distinctions … (contd..)

Software processes are not repetitive in the same sense as a manufacturing process:

driven by the changing “constraints of operation”.

In a manufacturing process:– the same controlled process is applied repetitively on raw materials yielding a finished product,– controlled variation of input parameters to optimize the yield or quality of the output is possible, – major change is uncommon; change occurs only when new machinery, or new material processing requirements, or new products are required.

For software process improvements to be effective, it is important to identify the “pain-points” and isolate the “real root-cause” – which is the difficult part (controllable factors)








Noteworthy distinctions … (contd..)A software process based on the CMMI is prone to numerous in-process and out-of-process variations



New Definition: Common cause variations, have a source of variation assignable to

something within the process control and,

whose impact can be isolated, corrected

and possibly eliminated with effective

“process management” practices.

New Definition: Special cause variations, have a source of variation assignable to

something outside the process control and,

whose impact can be resolved or minimized

with effective “project management”practices.

- Lack of domain understanding, - Internal churn resulting from people leaving midway, - Lack of budgets and resources, - Lack of appropriate tools

- Changing customer requirements, - Lack of access to the validation environment,- Interoperability issues with a legacy application,- Escape defects from the legacy







Six Sigma emphasizes process measurement and its relevance to the bottom-line performance with the onset of planning

Explicit strategies tying process performance to

bottom-line results or improvements, are not quite

evident in CMMI until the maturity level 4

Without Six Sigma thinking, at levels 2 & 3, it is likely

that frivolous measures are collected which might not

help to demonstrate improved business performance

A simple 4-step process inspired from Six Sigma can be applied at the CMMI level 2 itself

This helps to establish the right focus on

measurement and analysis at level 2









Widely accepted that “process quantification and process performance management” in CMMI, occurs only at levels 4 and 5

Can this focus shift downwards to the level 2?Note that, process measurement and analysis is addressed at level 2 itselfHowever, CMMI does not provide the required rigor of practices at level 2 for analysisHence, the 4-step process that–

defines the required analysis practices,serving as a strong foundation for high-maturity practice implementation,to improve the rigor of analyzing data,whereby, accelerating high-process maturity institutionalization



Quantitatively

Managed : Level 4

Initial : Level 1

Managed : Level 2

Optimizing

: Level 5

Defined : Level 3

0

1

2

3

4

5

Capability Levels

CMMI-DEV V1.2 Process Areas

Target Level

Achievement

Level






AGENDA

Establish the Context

What is the 4-step process?Establish that the process is stable & capableUnderstand the underlying process distributionFor a stable process identify the regressions and the residualsGeneralize the regression into a model

One Example of applying the 4-Step Process –

Conclusion, Q & A







Six Sigma as a problem solving methodology can

accelerate implementation of High-Maturity

practices of the CMMI constellationsWhen business goals are defined quantitatively, Six Sigma alongside CMMI has proven useful in minimizing the cycle time to transition maturity levelsA systematic and a consistent integrated approach of Six Sigma with CMMI can help in rapid acceleration of maturity levels from chaotic or level 1 mode to an optimizing or level 5 mode of operation in about 3-years timeframeWithout Six Sigma, it would typically takes anywhere between 12-18 months per level of maturity

(1 of 4)






Typical organizational contexts where such dramatic results were observed–

small- to medium-sized organizations or business units with less than 300 individuals in size,with annual staff attrition rates less than 5% with negligible impact to competency erosion,projects executed on mature domains/technologies with sufficient organizational memory of execution, good understanding of project requirements with an ability to manage requirements changes to within 10% effort deviation in the requirements-phasewhen at least every individual has a mandatory white-belt certification within the first 6 monthsupwards of 50% have either a green-belt or a black-belt certification

Establish the Context(2 of 4)







Essence of Six Sigma methodology in the CMMI context …

Uses the DMAIC approach (Define-Measure-Analyze-Improve-Control)by identifying business goals in quantitative terms,using voice of customer (VoC) that,establishes performance criteria or, the critical to quality (CTQ) requirements,adding value to the customer

Also ensures:Identification of critical business requirements that are in alignment with the organizational goals,by gathering voice of business (VoB) that, are critical to process (CTP)

(3 of 4)







Essence of high-maturity focus or mindset in the CMMI context can be summarized as …

No conclusions can be made without objective proof.The proof is in the data.There is no guarantee that a process change, contributes towards improvements unless verified and validated statistically.

(4 of 4)






What is the 4-step process?

Establish that the process is stable & capableUnderstand the underlying process distributionFor a stable process identify the regressions and the residualsGeneralize the regression into a model







1. Establish that the process

is stable and capable–Key Point: every stable and capable process, can be made more stable and more capableUse control charts/time-series charts to analyze outliersExample: 5-Whys to arrive at a process causeKey Point: When using sample data, consider the mean value of a large number of observations from independent homogenous projects Central Limit Theorem → normal distribution

(1 of 4)







2. Understand the underlying

process distribution–Key Point: Anderson-Darling test of normality is the most widely used test by statistical software (E.g., Minitab)High p-value shows no statistically significant departure from normalityNull hypothesis is that the data is normal – so a high p-value > 0.05 is indicative of a normal distributionKey Point: If the “statistical mean” of derived measures of a number of homogenous projects is used, there is no need for the normality test

(2 of 4)







Key Point: Every derived measure uses some combination of the basic metric dataKey Point: A derived measure can itself be a combination of several phase-wise factors (sub-process control)

(3 of 4)

3. For a stable process identify the regressions and the

residuals–Key Point: Regressions or, the factors are picked based on process knowledge, cost, risks and ease of measurementKey Point: Residuals are a measure of the error in a model; difference between a fit (predicted value) and the actually observed value







4. Generalize the regression into a model–

Key Point: Statistical models help to tie process inputs with the process output using statistically relevant data

Key Point: Models establish a structure using which, estimation and hypothesis testing are enabled

Key Point: Model can be deterministic or predictiveDeterministic: simple math equation: E = mc2

Probabilistic: Uses underlying probability distribution/density functions of random variables that defines the dependent variableExample: Z = A + P + F (for one sample of a project)What if there are 10 samples from 10 different projects?

For each project, Z = A + P + F may still be valid

Taken together, Z = 0.199 + 0.658*A + 0.789*P + 0.345 F+ eRegression model must be iteratively improved by fine-tuning it using the “actual” valueUsing large samples makes the regression equation better

(4 of 4)






One Example of applying the 4-Step Process –

Do we need this measure?Cost of Poor Quality : An important level 4 metric which is able to quantify the extent of rework






Do we need this measure?

Which basic metric and derived measure should we consider in the 4-step process?

Related to this question are the following questions –Why do we need to understand this measure? What do we do with this understanding once we have gained it?What type of a “process input” does this measure cater to? Is this a –

Controllable factor ?Standard Operating Procedure ?Noise ?

Key Point: The 4-step process applies only to a “controllable factor”; standard operating procedures and noise cannot be quantified reliably

(1 of 2)






Do we need this measure?(2 of 2)

To answer these questions meaningfully –A thorough design and understanding of the process is required,because, it is in the lifecycle phases that, time is consumed, errors or defects injected and detected

Key point: Measurement process should therefore be an integral part of the development lifecycle

Key point: Data must be collected and analyzed in real-time using simple techniques for it to be useful

Key point: There is little meaning in using software data analysis as in a typical post-mortem – after the fact






One Example of applying the 4-Step Process

Step 1: Establish that the process is stable and capable

Cost of poor quality (%): Ratio of the sum of internal and external failure fixing effort (major and minor fault) over total project effortEach observation is a rolled up data (mean) of different projects in execution after establishing data homogeneityFor outliers (point 4), determine cause using a structured 5-Whys approach

(1 of 4)







Step 2: Understand the underlying process distribution

Anderson-Darling test of normality is the most widely used test by statistical software Normality is required for continuous data prior to measuring the process capability Is the confidence interval for mean acceptable?

(2 of 4)

Variance measures

the degree to which

the probabilities of

a random variable

is spread out –

more the spread,

more the uncertainty

to predict or guess

the value.







Step 3: For a stable process identify the regressions and the residuals

(3 of 4)

Use One-way ANOVA for analysis of different phase-wise contributors

P-value is < 0.05; reject the null hypothesis that the mean COPQ % of the different phases are equalIdentify the regressions for improvements (CUT and Test)







Step 4: Generalize the regression into a model

(4 of 4)






A simple 4-step process which can be applied to –

establish an early focus on measurement and analysis of software data with a Six Sigma mindset,

based on emphasizing the identification of business objectives,

while considering the voice of business (CTP) and voice of customer (CTQ) was explored

This approach emphasizes identification of “pain-points” and the “real-root causes” to target the quantification and improvement effort (using controllable factors)

Special cause: with project management practices

Common cause: with process management practices

Conclusion






From the experience gained from small- to medium-organizations,

Six Sigma at the tactical level with,

model-based improvements using CMMI at the strategic level,

offers the benefit of cycle-time-reduction to advance in process maturity

This approach –

can target individual processes (continuous representation) or,

a group of process areas targeting practices involved in a maturity level (staged representation)

Conclusion






Six Sigma techniques relevant to a software process can be applied by organizations at any level of process maturity

Organizations that delay Six Sigma consideration until the level 4 may miss significantly useful opportunities to streamline improvement effort

Without a Six Sigma mindset, frivolous measures are collected which might not help to demonstrate improved business performance

May require to revamp their measurement program after attaining level 3

Conclusion






One of the most useful insights to take-away from this presentation–

every stable and capable process can be made more stable and more capable

start your high-maturity journey early, drivecarefully using the 4-step process and, reach safely!

Conclusion






References

[Chrissis 2011] Chrissis, Mary Beth et al, “CMMI for Development®: Guidelines for Process Integration and Product Improvement, Third Edition”, Addison-Wesley Professional. 2011

[Nandyal 2012] Nandyal, Raghav, “Building and Sustaining High-maturity Software Organizations”, Tata McGraw-Hill Education. 2012






DISCUSSIONS, Q&A

Raghav S. NandyalChief Executive Officer

SITARA Technologies Pvt. Ltd.#54, Sri Hari Krupa6th Main RoadMalleswaramBangalore KA 560 003Telephone: +(91-80) 2334-3222Mobile: + 984-523-3222Email: [email protected]

URL: http://www.SITARATECH.com

Thank You!

2180_Nandyal

Documents

manufacturing process

controlled process

process stability

step process

software development

software process improvements

context of process improvement

sigma application