Software Engineering Software Engineering Software Quality Assurance
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Software EngineeringSoftware Engineering
Software Quality Assurance
ObjectivesObjectives
1. To motivate for software quality assurance.
2. To provide a historical context for software quality assurance.
3. To consider the benefits of formal technical reviews.
4. To introduce statistical software quality assurance (SSQA).
Steps in Project PlanningSteps in Project Planning
Scope—understand the problem and the work that must be done. Estimation—how much effort? how much time? Risk—what can go wrong? how can we avoid it? what can we do
about it? Schedule—how do we allocate resources along the timeline?
what are the milestones? Control strategy—how do we control quality? how do we
control change?
SoftwareProject
Plan
Project ScopeEstimatesRisksScheduleControl strategy
Defining QualityDefining Quality
One goal of software engineering is to produce high-quality software. But what is “software quality?”
Software quality assurance (SQA) is an umbrella activity applied throughout the software process.
General engineering objective: reduce the “variation between samples.” But how does this apply to software?
Quality of Design: Characteristics that designers specify for an item. Encompasses requirements specification, analysis and design.
Quality of Conformance: The degree to which the design specifications are followed during manufacturing. Covers implementation.
Alternatively: User satisfaction = compliant product + good quality + delivery within budget and schedule.
McCall’s Triangle of QualityMcCall’s Triangle of Quality
PRODUCT TRANSITIONPRODUCT TRANSITIONPRODUCT REVISIONPRODUCT REVISION
PRODUCT OPERATIONPRODUCT OPERATIONCorrectness
Reliability
Usability
IntegrityEfficiency
Maintainability
Flexibility
Testability
Portability
ReusabilityInteroperability
Deriving Quality MetricsDeriving Quality Metrics
Each of McCall’s Quality Metrics combines different quality factors.
Examples:Correctness = Completeness + Consistency + Traceability
Portability = Generality + Hardware Independence + Modularity + Self-Documentation + System Independence
Maintainability = Conciseness + Consistency + Instrumentation + Modularity + Self-Documentation + Simplicity
“McCall’s quality factors were proposed in the early 1970s. They are as valid today as they were in that time. It’s likely that software built to conform to these factors will exhibit high quality well into the 21st century, even if there are dramatic changes in technology.”
Quality ConceptsQuality Concepts
Quality control: A series of inspections, reviews, tests to ensure that work products meet their requirements. Must include feedback.
Quality assurance: Analysis, auditing and reporting activities which communicate information about quality to management.
Cost of quality: all costs incurred in performing quality related activities. Prevention Costs (quality planning, formal technical reviews, test
equipment and training) Appraisal costs (trying to gain insight into product condition = in-process
inspection, equipment calibration and maintenance, testing) Failure costs (detect an error prior to shipping = rework, repair, failure
mode analysis) External failure costs (detect an error after shipping = complaint
resolution, product return and replacement, help line support, warranty work)
Why SQA Activities Pay Off?Why SQA Activities Pay Off?
100
10log
scale
1
anal.design
codetestsystem
testfielduse
0.751.001.50
3.00
10.00
60.00-100.00
Cost to find and fix a defect
Software Quality AssuranceSoftware Quality Assurance
“Conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics.”
FormalTechnicalReviews
SQA
Test Planning& ReviewMeasurement
Analysis&
Reporting
ProcessDefinition &Standards
SQA DefinitionSQA Definition
Aspects of SQA:1. Software requirements are the foundation from which quality is
measured. Lack of conformance to requirements is lack of quality.
2. Specified standards must be followed.
3. A set of implicit requirements often goes unmentioned (e.g. ease of use). If software conforms to explicit requirements but fails to meet implicit requirements, software quality is suspect.
History: Prior to 20thC province of the individual craftsman.
First formal quality assurance at Bell labs in 1916.
Software SQA parallels this (1950s and 1960s done by individual programmers, 1970s military introduced structured SQA)
SQA GroupSQA Group
Prepares an SQA plan for the project. This must identify evaluations, audits, reviews, standards, procedures for error reporting and tracking, and feedback.
Participates in the development of the project process. Reviews software engineering activities to verify
compliance with the defined software process. Audits designated software work products to verify
compliance with those defined as part of the software process.
Ensures that deviations in work products are documented and handled according to set procedures.
Records any non-compliance and reports to senior management.
Formal Technical Reviews Formal Technical Reviews
Formal technical reviews can be up to 75% effective in uncovering design flaws (which cover 50-65% of all errors)
“There is no particular reason why your friend and colleague cannot be your sternest critic” - Jerry Weinberg
Objectives:1. To uncover errors in function, logic, or implementation for any
representation of the software.
2. To verify that the software under review meets its requirements.
3. To ensure that the software has been represented according to predefined standards.
4. To achieve software that is developed in a uniform manner.
5. To make projects more manageable.
A Case for ReviewsA Case for Reviews
Defect amplification: a model for the generation and detection of errors during a sequence of tasks.
Example: Each test step uncovers 50% of errors but no reviews conducted in
earlier stages. 10 errors in analysis lead to 12 latent defects. Assume amplification of 1:1.5.
Formal technical reviews with 50-70% efficiency in analysis design and coding reduce latent defects to 3.
Errors from previous step
Errors passed through
Amplified errors 1:x
Newly generated errors
Defects Detection
Percent efficiency for error detection
Errors passed to next step
What Are Reviews?What Are Reviews?
What reviews are: a meeting conducted by technical people for technical people
a technical assessment of a work product created during the software engineering process
a software quality assurance mechanism
a training ground
What reviews are not: A project budget summary
A scheduling assessment
An overall progress report
A mechanism for reprisal or political intrigue
The PlayersThe Players
reviewleader
producer
recorder reviewer
standards bearer (SQA)
maintenance oracle
user rep
Conducting the Conducting the ReviewReview
be prepared—evaluate product before the review
review the product, not the producer
keep your tone mild, ask questions instead of making accusations
stick to the review agenda
raise issues, don't resolve them
avoid discussions of style—stick to technical correctness
schedule reviews as project tasks
record and report all review results
1.
2.
3.
4.
5.
6.
7.
8.
Metrics Derived from ReviewsMetrics Derived from Reviews
Inspection time per page of documentation Inspection time per KLOC or FP Inspection effort per KLOC or FP Errors uncovered per reviewer hour Errors uncovered per preparation hour Errors uncovered per SE Task (e.g. design) Number of minor errors (e.g. typos) Number of major errors (e.g. non-conformance to
design) Number of errors found during preparation
Statistical SQAStatistical SQA
ProductProduct& Process& Process
measurement
... an understanding of how to improve quality ...
• • collect information on all collect information on all defectsdefects• • find the causes of the find the causes of the defectsdefects• • move to provide fixes for move to provide fixes for the processthe process
Aspects of QualityAspects of Quality
Software Reliability: Important aspect of overall quality The probability of failure free operation of a computer program in
a specified environment for a specified time. Where failure is non-conformance to specification. Example: program X is estimated to have a reliability of 0.96 over
8 elapsed processing hours. Mean Time Between Failure (MTBF) = Mean Time To Failure
(MTTF) + Mean Time To Repair (MTTR). Regarded by some as better than num. defects.
Software Availability: Probability that a program is operating according to requirements
at a given point in time. Availability = [MTTF / (MTTF + MTTR)] * 100%.
Further Aspects of QualityFurther Aspects of Quality
Software Safety: Important in safety critical systems Unlike software reliability, failure leads to a hazard or
mishap. Use techniques from Risk Analysis. Hazards are
identified and categorized according to probability and criticality.
Example: Cruise control in a car. Hazards: (1) Causes uncontrolled acceleration, (2) does
not respond to depression of break pedal, (3) does not engage when switch is activated, (4) slowly loses or gains speed.
Mistake ProofingMistake Proofing
Poka-Yoke (Mistake Proofing) developed by Shigeo Shingo at Toyota in the 1960s.
A quality assurance technique applicable to software. Automatic error detection and prevention.
Devices: Prevention of potential quality problems before they occur. Rapid detection of quality problems if they are introduced.
Everyday examples: seatbelt warning signal (detection) or Car unable to start when in gear (prevention).
Characteristics: It is simple and cheap. Otherwise it will not be cost effective. It is part of the process. Integrated into the process. It is located near the process task where the mistakes occur. Allows
rapid feedback and error correction.
ISO9000 Quality StandardsISO9000 Quality Standards
A quality assurance system is defined as the organizational structure, responsibilities, processes and resources for implementing quality management.
Satisfy customers expectations by meeting their specifications.
ISO9000: A generic description of quality assurance activities applicable to
any business Adopted by many countries Often required to supply goods and services to the government Compliance scrutinized by third party auditors and subject to
semi-annual review
ISO9001: Standards that apply to Software Engineering
Related Documents
