8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
1/36
Leiden Institute of Advanced Computer Science
1
Systems Development and Project
Management Software quality assurance
Prof. dr. Thomas Bck
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
2/36
Leiden Institute of Advanced Computer Science Dates
Feb. 1 14:45 17:30 Introduction, Project DescriptionFeb. 2 13:45 16:30 STEP WISE Approach to Project Planning
Feb. 9 13:10 15:45 STEP WISE Approach to Project Planning,SAVE ENERGY Case
Feb. 15 14:45 17:30 Selecting an Appropriate Software Dev.
Approach
Feb. 16 15:15 18:00 Activity Planning and Resource Allocation
Feb. 22 14:45 17:30 Software Effort Estimation
Feb. 23 13:15 15:45 Risk management, project escalation
Mar. 1 14:45 17:00 ExamMar. 2 13:45 16:30 Risk Management, Project monitoring and
control
Mar. 8 14:45 17:30 Software Quality Assurance
Mar. 9 13:45 16:30 Managing People; Contract Management
Mar. 18 15:00 17:00 Trade Fair2
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
3/36
Leiden Institute of Advanced Computer Science
3
STEP WISE overview
9. Execute plan
10. Lower level
planning
0.Select
project1. Identify
project objectives
2. Identify project
infrastructure
3. Analyze
pr. characteristics
4. Identify products
and activities
5. Estimate effort
for activity
8. Review/ publicize
plan
6. Identify activityrisks
7. Allocate
resources
Review
Lower
level
detailFor each
activity
Some relate to qualities
Installation standards,procedures.
Special quality requirements ?
Entry, exit, process requirements ?
Review of overall quality aspects.
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
4/36
Leiden Institute of Advanced Computer Science
4
Software quality
Of increasing concernE.g. safety critical systems, dependence on core
ISProject control concerns:
Need to make project progress visible
Every task has a deliverable
Errors accumulate with each stageErrors become more expensive to remove thelater they are found
It is difficult to control the error removal process
(e.g. testing)
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
5/36
Leiden Institute of Advanced Computer Science
5
Software quality
Three Specifications:Functional: What the system is to do.
Quality: How well the functions are to operate.Resource: How much is to be spend on the system.
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
6/36
Leiden Institute of Advanced Computer Science
6
ISO 9126 software qualities
Functionality Does it satisfy user needs?
Reliability Can the software maintain itslevel of performance?
Usability How easy is it to use?
Efficiency Relates to the physical resourcesused during execution
Maintainability Relates to the effort needed tomake changes to the software
Portability How easy can it be moved to anew environment?
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
7/36
Leiden Institute of Advanced Computer Science
7
ISO 9216
Defined in 1991
To tackle the question of definition of software
quality
Also suggests sub-characteristics of the main
ones outlined here (outside main standard)
See next slides
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
8/36
Leiden Institute of Advanced Computer Science
8
Functionality sub-characteristics
Suitability
Accuracy
InteroperabilityAbility of software to interact with other software components
Compliance
Degree to which software adheres to application-related
standards or legal requirements, e.g. auditSecurity
Control of access to the system
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
9/36
Leiden Institute of Advanced Computer Science
9
Reliability sub-characteristics
Maturity
Frequency of failure due to faults - the more the
software has been used, the more faults will havebeen uncovered and removed
Fault-tolerance
Recoverability
Note that this is distinguished from security-see above
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
10/36
Leiden Institute of Advanced Computer Science
10
Further quality sub-characteristics
Usability sub-characteristics:
Understandability: easy to understand?
Learnability: easy to learn?
Operability: easy to use?
Efficiency sub-characteristics:
Time behavior, e.g. response timeResource behavior, e.g. memory usage
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
11/36
Leiden Institute of Advanced Computer Science
11
Further quality sub-characteristics
(contd)
Maintainability sub-characteristics:
Analyzability: ease with which the cause of a
failure can be foundChangeability: how easy is software to change?
Stability: low risk of modification having
unexpected effects
Testability
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
12/36
Leiden Institute of Advanced Computer Science
12
Quality sub-characteristics (contd)
Portability sub-characteristics:
Adaptability
Installability
Conformance: standards that have bearing on
portability (compare to compliance) - e.g. use of
high-level language
Replaceability: factors giving upwardscompatibility - downwardscompatibility is
excluded
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
13/36
Leiden Institute of Advanced Computer Science
13
Quality relationships
Indifferent
One has no effect on the other
Competitive
A system can only be good in respect to one
quality at the expense of another
ComplementaryA system which is good in respect to one quality islikely to be also good in respect to the other
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
14/36
Leiden Institute of Advanced Computer Science
14
External qualities:
Changeability
Testability
Internal vs. external qualities
Internal qualities (SQCs):
Modularity
GeneralityExpandability
Self-descriptiveness
SimplicityModularity
Instrumentation
Self-descriptiveness
Translate into
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
15/36
Leiden Institute of Advanced Computer Science
15
External qualities:
Portability
Internal vs. external qualities (contd)
Internal qualities (SQCs):
Modularity
Self-descriptivenessMachine independence
Software system
independence
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
16/36
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
17/36
Leiden Institute of Advanced Computer Science
17
Map measurement ontoratings scale to show
degree of satisfaction:
!"#$%"(#")*+
!,-.&/# (012+
!" $
"%$ &
'%() *
((%($ "
('%") (
+") )
Using ISO 9126 quality standards
(contd)
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
18/36
Leiden Institute of Advanced Computer Science
18
!"#$%&' )*+,-./0 1#2 3#&%.4 152 6/,-0 1# 7 52
!"#$%&$#$'( * + ,*
-..$/$"0/( 1 * ,2
34%&$#$'( 5 5 ,6
8,$ 9:
Using ISO 9126 quality standards
(contd)
Work out how ratings are to be combined
ISO 9126 does not specify how to do that only that
some method must be devised
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
19/36
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
20/36
Leiden Institute of Advanced Computer Science
Quality specification
Each project has three sets of requirements
Functional requirements: what the system is to do
Quality requirements: how well it is to do it
Resource requirements: how much it is going to
cost
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
21/36
Leiden Institute of Advanced Computer Science
21
Quality specification, e.g. ease of
installation
Definition of attribute
The amount of effort needed to install the package
for a new customer
Measurement scale
Hours
How testedTime needed to install system at three differentsites
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
22/36
Leiden Institute of Advanced Computer Science
22
Worst acceptable limit
4 hours
Planned limit
1 hours
Best achievable
30 minutes
Quality specification, e.g. ease of
installation(contd)
Define these for user-friendliness!
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
23/36
Leiden Institute of Advanced Computer Science
23
How do we achieve product quality?
The problem: quality attributes tend to beretrospectivelymeasurable
Need to be able to examine processes bywhich product is created beforehand
The production process is a network of sub-processes
Output from one process forms the input tothe next
Errors can enter the process at any stage
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
24/36
Leiden Institute of Advanced Computer Science
24
Product vs. process quality management
Errors are more expensive to correct at later
stages
Need to rework more stages
Later stages are more detailed and less able to
absorb change
Barry Boehm
Error typically 10 times more expensive to correct
at coding stage than at requirements stage
100 times more expensive at maintenance stage
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
25/36
Leiden Institute of Advanced Computer Science
25
For each activity, define!
Entry requirements
These have to be in place before an activity can
be startedExample: a comprehensive set of test data and
expected results be prepared and independently
reviewed against the system requirement before
program testing can commence
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
26/36
Leiden Institute of Advanced Computer Science
26
For each activity, define!
(contd)
Implementation requirements
These define how the process is to be conducted
Example: whenever an error is found andcorrected, alltest runs must be completed,
including those previously successfully passed
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
27/36
Leiden Institute of Advanced Computer Science
27
For each activity, define!
(contd)
Exit requirementsAn activity will not be completed until these
requirements have been metExample: the testing phase is finished only whenall tests have been run in succession with nooutstanding errors
Software quality planThese requirements may be laid down in sitestandards, or a quality plan may be drawn up for aspecific project
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
28/36
Leiden Institute of Advanced Computer Science
28
Inspections general principles
When a piece of work is completed, copies
are distributed to co-workers
Time is spent individually going through thework noting defects
A meeting is held where the work is then
discussedA list of defects requiring re-work is produced
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
29/36
Leiden Institute of Advanced Computer Science
29
Inspections advantages of approach
An effective way of removing superficial
errors from a piece of software
Motivates the software developer to producebetter structured and self-descriptive code
Spreads good programming practice
Enhances team-spiritThe main problem maintaining the
commitment of participants
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
30/36
Leiden Institute of Advanced Computer Science
30
General movement to give software more
quality
Increase the visibilityof software
Put methodinto processes of development
Check intermediate stages
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
31/36
Leiden Institute of Advanced Computer Science
31
External standards ISO 9001:2000
Ensure that a monitoring and control system to checkquality is in place
Only certification of development processNot software development-specific
Main activities:
Determine customer needs and expectation
Establish quality policyDesign product creation process with responsibilities
Measure effectiveness and efficiency
Take corrective action
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
32/36
Leiden Institute of Advanced Computer Science
32
ISO 9001:2000 (Criticism)
Expensive, time consuming
Putting smaller firms at a disadvantage
Preoccupation with certification
Can distract attention from real problems of
producing quality products
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
33/36
Leiden Institute of Advanced Computer Science
33
External standards Capability Maturity
Model
Levels of Process Maturity:Level 1 Initial: haphazard procedures followed
Any organization at this level by default
!
Level 2 Repeatable: basic project managementprocedures
The way individual tasks are carried out will dependlargely on person doing it.
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
34/36
Leiden Institute of Advanced Computer Science
34
External standards Capability Maturity
Model
Levels of Process Maturity:Level 3 Defined: how should each task in the
softw. Development life cycle be doneLevel 4 Managed: products and processes aresubject to measurement and control
Level 5 Optimizing: improvements based onmeasurement data
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
35/36
Leiden Institute of Advanced Computer Science
35
External standards Capability Maturity
Model (contd)
Level Key Process Areas
Initial Not applicable
Repeatable Configuration management, qualityassurance, project planning, etc.
Defined Peer reviews, integrated softwaremanagement, training program, etc.
Managed Quality management, processmeasurement and analysis
Optimizing Process change management,technology innovation, defect prevention
Software Configs,
i.e., Version Control
8/13/2019 Sdpm Lecture8 Softwarequalityassurance 111021041202 Phpapp01
36/36
Leiden Institute of Advanced Computer Science
Summary
Quality = vague concept. Requirements have to becarefully defined.
There have to be practical ways to test relativepresence / absence of a quality.
Most qualities can only be tested when system is
completed.
We need ways of checking during development.Some procedures focus on testing products, others on
evaluating quality of the development process.