1 W R Systems, Ltd. 1 Integrating Quality Assurance into the Software Development Life Cycle Leslie Tierstein, STR LLC Hilary Benoit, W R Systems Overview (1) z Why bother with QA? z QA and the SEI CMM/CMMI z Defining the Software Development Process z Setting up the QA Function z Selecting the Pilot Project
31
Embed
Integrating Quality Assurance into the Software ...
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
W R Systems, Ltd. 1
Integrating QualityAssurance into the SoftwareDevelopment Life Cycle
Leslie Tierstein, STR LLCHilary Benoit, W R Systems
Overview (1)z Why bother with QA?z QA and the SEI CMM/CMMIz Defining the Software Development
Processz Setting up the QA Functionz Selecting the Pilot Project
2
Overview (2)z Tools, Procedures and Activitiesz Lessons Learnedz The next step - from pilot project to all
projectsz Summary
What is Quality?
z Quality - “The totality of features andcharacteristics of a product or service thatbears on its ability to satisfy givenfeatures.” (American Society for Quality,1978)
3
What is Quality Assurance?
z Quality Assurance - “consists of all theplanned and systematic activitiesimplemented within the quality systemthat can be demonstrated to provideconfidence that a product or service willfulfill requirements for quality.”
Why Bother with QA?
z Need to produce quality software productsin a repeatable and consistent manner
z Checks and Balancesz Customer Assurancez Carnegie Mellon’s Software Engineering
Institute’s Capability Maturity Model (SEICMM) - requires Software QualityAssurance (SQA)
4
SEI CMM and CMMI
z Model to gauge the maturity of thesoftware development process
z Superceded by CMM Integration (CMMI),incorporating ISO-9000 principles
z Software Process frameworky Five maturity levelsy Key Process Areas (KPAs)
SEI CMM Maturity Levels
z Level 1 - Ad hoc (chaotic)z Level 2 - Repeatable (disciplined)z Level 3 - Defined (standard; consistent)z Level 4 - Managed (predictable)z Level 5 - Optimizing (continuously
CMMI Maturity Level Key Process Areas 2 Managed - Planned Performed Managed Controlled
• Configuration Management • Process and Product Quality Assurance • Supplier Agreement Management • Project Monitoring and Control • Project Planning • Requirements Management
CMM/CMMI KPAsCMM Maturity Level Key Process Areas 3 Defined -
Standard Consistent
• Peer Reviews • Inter-group Coordination • Software Product Engineering • Integrated Software Management • Training Program • Organization Process Definition • Organization Process Focus
CMMI Maturity Level Key Process Areas 3 Defined -
Consistent across the organization
• Verification • Integrated Project Management • Requirements Development • Technical Solution • Product Integration • Organizational Training • Process Definition and Process Focus
7
CMM/CMMI KPAsCMM Maturity Level Key Process Areas 4 Managed
Predictable
• Software Quality Management • Quantitative Process Management
CMMI Maturity Level Key Process Areas 4 Quantitatively
Managed
• Quantitative Project Management • Organizational Process Performance
z Measures to quantify quality, process, andimprovements
CMM/CMMI KPAsCMM Maturity Level Key Process Areas 5 Optimizing
CMMI Maturity Level Key Process Areas 5 Optimizing
Continuously improving
• Causal Analysis and Resolution • Organizational Innovation and Deployment
z Proactive measures to improve qualityz 4-5 organizations nationwide
8
Define and Document theDevelopment Process
z Software development process is thefoundation to the QA process
z Should be:y well-definedy simpley clear phasesy entry and exit criteria
Software DevelopmentProcess/Methodology
z Strategyz Analysisz Designz Build and Testz Deployz Maintain
9
Define and Set up the QAFunction (1)
z Purpose and Goalsy Control cost, schedule, qualityy “Time box” of development
z Activities - vary with life cycle phasey QA <> Testing
z How to staff?y Programmers or non-programmers
z Skills required
Define and Set up the QAFunction (2)
z Resourcesy Corporatey Per project
z Independent Organizationz Management Support
10
Select the Pilot Project
z Oracle full life cycle development projectz Oracle Designer/Developerz Client-Server - Windows and HP-UNIXz Government contract - customer
requirement to achieve SEI CMM Level 2z Opportunity to integrate software quality
assurance into the full life cycle
Integrate QA into LifeCycle Phases
z Phase entry and exit criteria - inputs andoutputs
z Quality Checkpointsz Audits and reviews of products and
processesz Timely management notification of
problems - Risk Management
11
Strategy Phase
RFP/SOW IdentifyStrategy
TechnicalReview
QAAudit
CustReview
ANALYSISSTRATEGY
MGT/CM/QAPlans
QA and the Strategy Phase (1)
z Develop the QA Plan and Proceduresy MIL-STD-498y ISO-9000
z Create QA recordsz Determine Metricsz Review and Analyze Requirementsz Establish the Deliverable Review Process
12
QA and the Strategy Phase (2)
z Project Standards and Proceduresy Shared components and their managementy Externally developed coding standardsy Internally developed standards and
procedures
Tools and Techniques
z QA Records - Word templatesz QA Activities Tracking System (QATS)z Deliverable Review Route Sheetsz Quality Control Reportsz Requirements Traceability Matrix (RTM)z Checklists and Forms
13
Keep QA records
z Document all QAreviews andaudits
z Audit trail ofactivity
z Metricsz Sample form
Quality AssuranceTracking System (QATS)
z Databasez Reportsz Metricsz QA activities
tracking
14
Deliverable Review Process
Perform
Technical Review
PerformRequired QA
Inspection
EstablishConfigurationManagement
Baseline
Client Review
Enter Next Phaseor Perform
Corrective Action
If Approved
If Approved
If Approved
Deliverable Review RouteSheet (Sample)
Project Deliverable Route Sheet
Project: Files of Task #: Deliverable: Date: Return To:
Return To: ________________________________ ________________________________ ________________________________ _________________
INITIAL REVIEW FOLLOW UP REVIEWReport Name Date
SubmittedDateCompleted
Reviewer Follow-uprequired?
DateSubmitted
DateCompleted
Reviewer
User GuideP:\PROJ\...\Task2E\_____\DRAFT\____UG___.doc
Comments:
INITIAL REVIEW FOLLOW UP REVIEWMODULE DEFINITION form report
Module Name DateSubmitted
DateCompleted
Reviewer Follow-uprequired?
DateSubmitted
DateCompleted
Reviewer
Comments:
21
Unit Test ChecklistInitial Forms Module Checklist (Prior to Demo)
Module Name: Tester: <Tester name> Date of Test: mm/dd/yy Developer:
Tested Item Pass Fail
Tester Comments Priority Fixed Verified
Layout / Window Data Fields Color Size Tab position Labels and Titles Date Hints Required Fields Scrollbars Help Abbreviations Phone Numbers
Test Form (Sample)UNIT/UNIT INTEGRATION/BUG FIX TEST REPORT
Project: S/W Version:
Module ID & Short Name: Date:
SPR #(s) associated with this test:
SPR(s) attached: YES pp NO ppSPR #(s) received by phone: p p
If there is no bug/ SPR associated with this module,please check below, as appropriate.
This is an Enhancement ppThis is New Development pp
Associated LCCB #(s) - if applicable:
Test Site: WRS p p Customer ppTest Plan: Software Test Plan(ELIN 1010)
Developer Tester(s): TEST database : HP755 p p T600 ppPRODUCTION database: T600 ppCheck at least one for QA to test in
Brief Description/Purpose of Test: Attach appropriate Bug Reports, if possible.
Other affected modules (for unit integration/impact analysis) These include other screens or reports that this change will affect and that therefore will need to be tested.
Other associated packages, functions, procedures and views: Specify ALL that need to accompany this test.
Test Complete? Yes p p No pp Test Result: Pass p p Fail pp
Results/Comments:
Item/Action to be Tested (complete on separate sheet, if necessary) Result
Does the User Documentation require updating as a result of this bug fix/enhancement? YES pp NO ppQA Signature & Date:
CM InformationVersion of module:
Date moved to T600-TEST: Date moved to T600-PRODUCTION:
Test Form
z Formalz “with a form”z with a review
and approvalprocess
22
Problem Tracking (1)
Problem Tracking (2)
23
QA and the Build and TestPhase (2)
z Integration Test - partially automatedy Business scenarios via QA/Directory Load Runner for load testing
z System Testy Customer/client involvementy Acceptance test
z Integrated Project Teams (IPT)z Independent Validation and Verification
(IV&V)
A Test Data Entry Screenin QA/Director
24
Entering Bugs intoQA/Director
QA and the DeploymentPhase
z Phased implementation (no “Big Bang”)y By function/subsystemy By organization/user group
z QA reviews and test procedures continuedz Expedite test and delivery of modified
code - fast turnaround requiredz User training - review and test of training
materials
25
QA and the MaintenancePhase
z Continue with established QA and CMproceduresy Action Items/User meetingsy MoSCoW evaluation and followupy Problem Reports - user accessible
Collect Project Metrics
z Areas of greatest problems/defectsz Number/results of QA audits and reviewsz Test coverage and test resultsz Problem Reports/Defects found - e.g., per
module, per subsystems, classification andtype, time taken to resolve
z Development Time - Estimated vs. Actualz - SEI CMMI Level 4 Quantitatively Managed
26
Process Improvement
z Take existing processz Analyze step-by-stepz Modify to improve
y e.g., testing/QA/CM processy unit testing - formalized
z Training - e.g., COEs, Test Writing, Testing
z - SEI CMMI Level 5 Optimizing
Lessons Learned
z Acceptance of QAz What workedz What didn’t work
27
Acceptance of QA
z QA function - perceived as “value added”z Not confrontational/criticalz Provide guidance, oversight, trainingz Assistance in process improvementz Well-designed QA Plan and proceduresz Concrete activities and reportsz QA Schedulez Part of the team
What Worked
z Formal Review process of deliverablesz Strong Requirements Management and
RTMz Collaboration of QA with TM and CMz Participation of QA in meetingsz QA sign-off in Testingz Formal bug trackingz Peer Reviews
28
What did NOT work
z Excessive paperwork for developersz Anything causing lengthy turnaround on
deliverablesz Expecting developers to read lengthy
standards documentsz Assuming developers would enter all
required RTM informationz Informal Unit Testing
Implementing QA on allProjects
z “Clone” the processz Use successful “artifacts”z Target trainingz Use “Lessons learned”z Expand the SQA group
29
Summary: QA activities tointegrate into the SDLC (1)
z Scheduled audits & reviews of all project processesand deliverables
z Maintenance of QA records of reviews and auditsz Management notification of non-compliance with
standards and procedures, or of notable problemsz Resolution of Problem Reports and Verification of
corrective action Manage the Requirements’Traceability process
Summary: QA activities tointegrate into the SDLC (2)
z Managing the Requirement Traceability processz Peer Reviews, code walkthroughsz Including QA personnel in project and customer
meetingsz Providing training in standards, testing, or other
QA-related topicsz Independent Testing
30
Summary of Steps
z Define and Document the Software DevelopmentProcess
z Verify/Obtain support of Top Managementz Set up the QA functionz Select the Pilot Projectz Integrate QA activities into the development life
cycle phasesz Use Lessons Learned to implement QA on other
projectsz Expand QA group function, as required
Conclusion
z Successful deployment of pilot projectz Integration of software quality assurance
into the life cyclez SEI CMM - Level 3 compliant
31
Quality-Related Web Sites
z www.asq.org - American Society for Quality (ASQ)z www.iqa.org - Institute of Quality Assurancez www.iso.ch - International Organization for
Standardization (ISO)z www.nist.gov - National Institute of Standards and