Software Project Management Lecture # 14. Outline Six Sigma Software Reliability Failure Measures of Reliability & Availability Software Safety Quality.

Software Project Management

Lecture # 14

Outline

Six Sigma Software Reliability Failure Measures of Reliability & Availability Software Safety Quality Standards SQA Plan

Six Sigma …

It is the most widely used strategy used in industry for statistical quality assurance.

It was originally popularized by Motorola in 1980s.

It can be described as A rigorous and disciplined methodology that

uses data and the statistical analysis to measure & improve a company’s operational performance by identifying and eliminating ‘defects’ in manufacturing & service-related processes

Six Sigma (Contd.)

Six sigma methodology defines 3 core steps: Define customer requirements, deliverables, &

project goals via well defined methods of customer comm.

Measure the existing process & its output to determine current quality performance (collect defect metrics)

Analyze defect metrics & determine the vital few causes.

Six Sigma (Contd.)

If an existing software process is in place, but improvement is required, Six Sigma suggests 2 additional steps: Improve the process by eliminating the root

causes of defects Control the process to ensure that future work

does not reintroduce the causes of defects These core steps and additional steps are

also referred to as DMAIC method

Six Sigma (Contd.)

If an organization is developing a software process (rather than improving an existing one), the core steps are augmented by: Design the process to (1) avoid the root causes

of defects and (2) to meet the customer requirements

Verify that the process model will, in fact, avoid defects and meet customer requirements

This is referred to as DMADV method

Software Reliability

Definition The probability of failure free operation of

a computer program in a specified environment for a specified time.

Reliability is the probability of not failing in a specified length of time

Software Reliability (Contd.)

Mathematical representation F(n) = 1 - R(n)

Where, R(n) = probability of reliability (i.e. not failing) n = no. of time units,

if time unit is assumed in days then probability of not failing in 1 day is R(1)

F(n) = probability of failing in a specified length of time

Software Reliability (Contd.)

It is a quality factor that can be directly measured and estimated using historical ‘development data’.

It measures how often s/w encounters a data input or other condition that it does not correctly process to produce correct answer

If programX has reliability of 0.96 (over 8 processing hours) then it means, if programX runs 100 times – it will operate correctly 96 times

Failure

Non-conformance to s/w requirements leads to failures

Negative results or in worst case no output is failure

Some failures can be corrected in seconds, some in weeks and others in months

One failure may introduce other errors (in effect other failures)

Measures of Reliability & Availability

Early work in software reliability attempted to extrapolate the mathematics of hardware reliability theory to prediction of software reliability. But, Most hardware reliability models have predicted

on failure occur due to physical wear (corrosion effects, shock, temperature, etc.) rather than design defects.

The opposite is true for softwares. All software failures can be traced to design or implementation problems.

Measures of Reliability & Availability Measure of Reliability

Consider a computer-based system. A simple measure of reliability for such a system is mean-

time-between-failure (MTBF) MTBF = MTTF + MTTR

MTTF = Mean-time-to-failureMTTR = Mean-time-to-repair

Many researchers argue that MTBF is more useful term than defects/KLOC or defects/FP as user is more concerned with failure rate as compared to defect count.

Each defect does not have same failure rate and the total defect count gives little indication of the reliability of a system

Measures of Reliability & Availability

Measure of Availability Software availability is the probability that

a program is operating according to requirements at a given point in time.

It is defined asAvailability = [MTTF / (MTTF + MTTR] * 100%

Availability measure is sensitive to MTTR

Software Safety

This SQA activity focuses on identification & assessment of potential hazards that may affect software negatively & cause an entire system to fail.

Early identification of hazards can help to have design features that either eliminate or control potential hazards.

A modeling & analysis process is conducted as part of s/w safety. Initially – hazards identified & categorized by criticality &

risk Next – analysis techniques are used to assign severity &

probability of occurrence (similar to risk analysis methods but different as the emphasis in this case is on technology issues rather than project )

Software Safety (Contd.)

The following analysis techniques can be used: Fault tree analysis Real-time logic Petri Net model

After hazards identification & analysis, the next step is to specify safety related requirements, i.e., to find A list of undesirable events & the desired system

responses to these events Role of s/w in managing undesirable events is then

indicated

The ISO 9000 Quality Standard

A quality assurance system may be defined as the organizational structure, responsibilities, procedures, processes, & resources for implementing quality management.

ISO 9000 describes a quality assurance system in generic terms that can applied to any business regardless of the products services offered.

Getting Certified…

To become registered to one of the quality assurance system models contained in ISO 9000, a company’s quality system & operations are scrutinized by third-party auditors for compliance to the standard for effective operation.

Upon successful registration, the company is issued a certificate from a registration body represented by the auditors.

Semi-annual surveillance audits ensure continued compliance to the standard.

ISO 9001:2000

ISO 9001:2000 is the quality assurance standard that applies to software engineering.

The ISO 9001:2000 standard contains 20 requirements.

For a software organization to become registered to this standard, it must establish policies & procedures to address each of these requirements

As ISO 9001:2000 standard is applicable to all engg. disciplines, a special set of ISO guidelines (ISO 9000-3) have been developed for use in software process.

Quality Standards

Reading Assignment CMM CMMI

The SQA Plan Provides a roadmap for establishing SQA

Developed by SQA group (or software team if SQA group does not exist)

A standard structure for SQA plans by IEEE recommends the following:

1) Scope & purpose of the plan.2) Description of all s/w engg. work products that

fall within range of SQA.3) All applicable standards & practices that are

applied during the software process.

The SQA Plan

4) SQA actions & tasks (including reviews & audits) and their placement throughout the software process.

5) Tools and methods that support SQA actions & tasks.

6) Software configuration management procedures for managing change.

7) Methods for assembling, safeguarding, and maintaining all SQA-related records.

8) Organizational roles and responsibilities relative to product quality