Thailand SPIN: QA Excellence Workshop

QUALITY ASSURANCE EXCELLENCE Software Process Improvement

Nopparat Slisatkorn Prapasri Kajorndech Wittaya Grasaenak Thomson Reuters 5 August 2015

Instructors

2

Bee

Process

Technical

Specialist

Ton

Lead Software

Engineer

Kae

Lead Quality

Assurance

Engineer

Outline

Quality

CMMI overview

Process areas for ensuring the quality

5 Whys and Fishbone

Lean concept & Waste in Software Development

Value Stream Mapping

Summary

Q&A

3

4

Quality Definition

• Error-free

• Meet requirement

• Nothing more or less than the perception the customer has of you, your products, and your

services

• Value products and/or services delivered

• At the right time

Source: http://www.qualitydigest.com/html/qualitydef.html

Quality is a journey, not a destination.

5

QA & QC

6

“Quality Assurance is process oriented and focuses on

defect prevention, while quality control is product oriented

and focuses on defect identification.”

http://www.diffen.com/difference/Quality_Assurance_vs_Quality_Control

Introduction to CMMI

CMMI:

Capability Maturity Model Integration by SEI (Software Engineering Institute)

Capability

the ability to do something

Maturity

the condition of growing

Model

framework or direction to do something repeatedly

Integration

combining things together

“A framework that aid an

organization in improving processes

and its abilities to develop, acquire,

and maintain its products and

services”

7

CMMI Representatives

Stage representation

(Maturity Levels)

0

1

2

3

PA 1 PA 2 PA 3

Continuous (Target Profile)

Continuous representation

(Capability Levels)

8

Workshop to understand CMMI Process areas

Village Construction

Instruction

• Village – Theme colorful

• At least 3 houses completed in phase

• Customer will inform more requirements

• Sales price per quality house: $100

• Late deliver charge per house $30

• Incomplete village cannot be sold.

• Materials that you can buy

– Magazine paper

– A4 paper

– Scissor

– Cutter

– Pen color

– Ruler

– Post its

– Pencil

10

Timeframe

• Discuss on MOW, Goal and Buy materials – 5 minutes

• Develop and Deliver – 10 minutes

• Discuss Problem & How to improve – 5 minutes

11

Goals

http://www.mymagneticblog.com/

12

CMMI DEV - Process Group

Process Group Process Area CMMI DEV Maturity Level

Engineering Product Integration (PI) 3

Engineering Requirements Development (RD) 3

Engineering Technical Solution (TS) 3

Engineering Validation (VAL) 3

Engineering Verification (VER) 3

Process Management Organizational Process Definition (OPD) 3

Process Management Organizational Process Focus (OPF) 3

Process Management Organizational Training (OT) 3

Process Management Organizational Process Performance (OPP) 4

Process Management Organizational Performance Management (OPM) 5

Project Management Requirements Management (REQM) 2

Project Management Project Monitoring and Control (PMC) 2

Project Management Project Planning (PP) 2

Project Management Supplier Agreement Management (SAM) 2

Project Management Integrated Project Management (IPM) 3

Project Management Risk Management (RSKM) 3

Project Management Quantitative Project Management (QPM) 4

Support Configuration Management (CM) 2

Support Measurement and Analysis (MA) 2

Support Process and Product Quality Assurance (PPQA) 2

Support Decision Analysis and Resolution (DAR) 3

Support Causal Analysis and Resolution (CAR) 5

13

CMMI DEV – Maturity Levels

Maturity Level Process Area CMMI DEV Process Areas

2 Requirements Management (REQM) Project Management

2 Project Monitoring and Control (PMC) Project Management

2 Project Planning (PP) Project Management

2 Configuration Management (CM) Support

2 Measurement and Analysis (MA) Support

2 Process and Product Quality Assurance (PPQA) Support

2 Supplier Agreement Management (SAM) Project Management

3 Organizational Process Definition (OPD) Process Management

3 Organizational Process Focus (OPF) Process Management

3 Organizational Training (OT) Process Management

3 Integrated Project Management (IPM) Project Management

3 Risk Management (RSKM) Project Management

3 Decision Analysis and Resolution (DAR) Support

3 Product Integration (PI) Engineering

3 Requirements Development (RD) Engineering

3 Technical Solution (TS) Engineering

3 Validation (VAL) Engineering

3 Verification (VER) Engineering

4 Organizational Process Performance (OPP) Process Management

4 Quantitative Project Management (QPM) Project Management

5 Organizational Performance Management (OPM) Process Management

5 Causal Analysis and Resolution (CAR) Support

14

http://blog.optimalbi.com/2014/02/28/trust-me-would-i-lie-to-you/

15

http://www.lambertconsultinggroup.com/PDUEvent.aspx?e=175

16

Requirement Development Process

Collected

Requirement

Elicit stakeholder

needs, constraints

and interfaces

Prioritize

requirements

Peer review

requirements

Develop Product

requirements (component/functional/

non-functional)

Agreed

Requirement

17

User Story

As a Line user,

I want to create group chat

so that I can chat with many friends at the

same time.

Independent Negotiable Valuable Estimatable Small Testable

18

Acceptance Criteria

• After click creating group, the group name will be active within 1

seconds.

• The created group chat will be shown on Friends page under group

category.

• The number of participants in group will be shown behind the group

name.

• The maximum number of participants in each group is 50 persons.

• If the name of group hasn’t defined, the group name will be shown

as “no name”.

Specific Measurable Achievable Relevant Time-bound

19

Book Ordering Online Workshop Create User Story and Acceptance Criteria

Are you…?

21

Peer Review Process(Verification)

Work product

Plan/Prepare for

Peer Review

Conduct

Peer review

Evaluate

Peer Review

Analyse and

correct defect

Example:

When, Who, Which, How

Meeting invitation

Work product readiness Revised work

product

Evaluate result

Inspection

Team review

Walkthrough

Pair programming

Pair testing

Email

Peer Review Types

Formal

Informal

22

23

http://www.isixsigma.com/industries/software-it/defect-prevention-reducing-costs-and-enhancing-quality/

Book Ordering Online Peer Review Workshop Peer Review User Story and Acceptance Criteria

Cost of Quality

http://www.precisetestingsolution.com/betatesting.php

25

Cost & Feedback

26 http://www.agilemodeling.com/essays/modelReviews.htm#

Cost of Defect

http://support.smartbear.com/articles/codecollaborator/roi-calculator/

3

10 10

0

2

4

6

8

10

12

Peer review QA Customer

Cost to fix/Defect

27

Real Case Cost of Defect

http://www.cmcrossroads.com/article/case-peer-review?page=0%2C0

28

Real Case Cost of Defect -2

(Source: IBM Systems Sciences Institute)

29

30

Causal Analysis and Resolution

Opportunity

Outcome

Identify

opportunity

Analyse

opportunity

Implement and

control solution

Define and select

solution

Improvement action

and evaluation result

31

5 Whys • 5 Why’s can be used to investigate a specific failure to find a problems real root cause.

Tips:

• Keep asking until get the root cause

• If that didn’t happen, would the problem go away?

32

http://www.educational-business-articles.com/5-whys.html

Fishbone diagram

• "Cause-and-Effect Diagram" or "Ishikawa Diagram"

• Kaoru Ishikawa – 1960s

• A graphical method

• Finding the root causes of an effect (-/+).

33

http://en.wikipedia.org/wiki/Ishikawa_diagram

Example

34

5 Whys / Fishbone Workshop • Project delay • Deliver project on time but have many post release defects reported. • Deliver project on time, test 100% coverage but customers do not want

the product.

Steps: 1. Define the Effect 2. Choose Categories 3. Brainstorm Possible Causes 4. Ask Why? to find the root causes 5. Investigate

Validation Process (VAL)

Requirement

Select product to

be validated

Prepare for

validation e.g. environment,

invitation, etc

Analyze validation

result Validate

Validation reports

Validation issues

Enchantment request

36

There are many validation methods.

Examples:

• Sprint Review

• Demo

• Pilot

• Test (e.g. EAP, Alpha)

• Simulation

37

Anything that use time, effort, space, or

money without adding customer value.

38

Software Development Waste • Partially done work

• Extra features

• Re-learning

• Hand-offs

• Delays

• Task Switching

• Defects

Note: there are some necessary waste e.g. process, learning

7 waste

39

Wastes are related

@ Mark Robinson

40

Lean Key concepts in Lean

41

Value Stream Mapping (VSM)

• Used to analyze and design the flow

• Used in Lean environments

• Toyota

The six steps are:

• Identify the actions taken

• Specify the calendar time over the actions

• Specify actual work was taking place and waiting time

• Specify repeat work and time

• Total up the average time working

42

VSM Example

Register

2

10

Measure

5

20

Department Adm.

1

30

Meet Doctor

5

120

Payment

2

30

Get Medicine

5

90 Duration

(min)

Actual time

(min)

• Actual time = 2+5+1+5+2+5 = 20 minutes

• Total time = 10+20+30+120+30+90 = 300 minutes

• Process efficiency = 20/300 = 6.67%

• Then you can investigate which one is waste that you can eliminate.

43

ศรราชกาวไกล สงยาทวไทยทางไปรษณย

44

http://www.thairath.co.th/content/424212

http://www.si.mahidol.ac.th/th/hotnews_detail.asp?hn_id=1562

ผศ.นพ.วศษฎ วามวาณชย ผอ ำนวยกำร รพ.ศรรำช เปดเผยวำ จำกจ ำนวนผปวยทรบกำรรกษำใน โรงพยำบำลศรรำชมจ ำนวนเพมขนอยำงตอเนองในแตละวน สงผลถงควำมแออด จนท ำใหเกดกำรรอคอยในกำรรบยำ ช ำระคำบรกำร กวำจะไดกลบบำนกหมดเวลำไปเกอบครงวน ตนจงมอบนโยบำยให รศ.นพ.เชดชย นพมณจ ำรสเลศ รองผอ ำนวยกำรรพ.ศรรำช ดำนบรกำรผปวยนอกและพฒนำคณภำพ คดหาวธท จะชวยใหผปวยกลบบานไวขน จากผลการวเคราะหระบบงานเพอลดความสญเปลาตามแนวคด Lean จงเกดโครงการ “ศรราชกาวไกล สงยาท วไทย ทางไปรษณย” ซงเรำมองวำ“ไปรษณยไทย” เปนหนวยงำนทมศกยภำพ ทจะชวยใหยำของเรำไปถงมอผปวยโดยเรว มคณภำพ และประสทธภำพ ดวยไปรษณยดวนพเศษในประเทศ หรอ EMS สงดวนทวไทย นบเปนทำงเลอกใหมทฝำยเภสชกรรม รพ.ศรรำช ไดรวมกบ บรษท ไปรษณยไทย จ ำกด อ ำนวยควำมสะดวกใหผปวย โดยท ภก.วชต ต งจตตพร หวหนำฝำยเภสชกรรม รพ.ศรรำช ประธำนคณะท ำงำนกำรจดสงยำทำงไปรษณย พรอมทมงำนเปนก ำลงส ำคญในกำรท ำงำน ดวยขนตอนกำรใหบรกำรแบบครบวงจร “หลงเขารบการตรวจรกษาและไดรบใบส งยาจากแพทยแลว ผปวยสามารถขอเขารวมรบยาทางไปรษณยตำมขนตอนดงน 1. ตดตอทเคำนเตอรใหบรกำรสงยำทำงไปรษณย (ตรงขำมเคำนเตอรออกใบนดผปวย) หนวยตรวจอำยรศำสตรชน 2 หอง 210 ตกผปวยนอก 2. ผปวยกรอกใบแจงควำมจ ำนงเขำรวมกำรจดสงยำทำงไปรษณยพรอมเขยนชอ ทอยทตองกำรใหจดสง 3. เภสชกรประเมนกำรใชยำของผปวย อธบำยยำทผปวยไดรบ และใหตำรำงกำรรบประทำนยำแกผปวย พรอมใหเอกสำรใบยนยนกำรรบยำทำงไปรษณยทมหมำยเลขพสด 13 หลก 4. ผปวยช ำระเงนคำยำ และคำจดสงทำงไปรษณย รำคำ 150 บำท 5. รบยำตำมทอยทระบ โดยจดสงยำทำงดวนพเศษในประเทศ หรอ EMS สงดวนทวไทย ภำยใน 1-3 วน ท ำกำร นบจำกวนทตรวจเสรจ” อยำงไรกตำม ผทจะเขำมำรบบรกำรจดสงยำทำงไปรษณย ตองไดรบกำรประเมนจำกเภสชกรในกำรใชยำวำสำมำรถใชยำไดอยำงถกตองและปลอดภย และเปนผปวยทสำมำรถควบคมอำกำรไดคงทแลว

Tool

People Process Product

/Service

Quality

PDCA

Feedback

Plan Do

Check Act

45

PDCA: Dr. W. Edwards Deming

Goal & Collaboration

46

47

Reference

• CMMI: http://www.sei.cmu.edu/cmmi/

• CMMI & Agile: http://resources.sei.cmu.edu/asset_files/TechnicalNote/2008_004_001_14924.pdf

• Agile: http://agilemanifesto.org/

• Test Maturity Model Integration: http://www.tmmi.org/pdf/TMMi.Framework.pdf

• Test Process: http://www.slideshare.net/tokarthik/Test-Process

• Risk Management: http://www.dummies.com/how-to/content/the-essentials-of-managing-risk-in-

your-project.html

• Project Management: http://www.projectmanagementdocs.com/project-planning-

templates/project-management-plan.html

48

Reference – Cont.

Lean:

• Waste and Trash

– http://www.slideshare.net/at2009qc/waste-and-trashing

• What does lean mean for software Testing?

– http://vimeo.com/30617475

Fishbone: http://www.bulsuk.com/2009/08/using-fishbone-diagram-to-perform-5-why.html

5 whys: http://www.bulsuk.com/2009/03/5-why-finding-root-causes.html

VSM: http://agileconsulting.blogspot.co.uk/2010/03/value-stream-mapping-vsm-is-lean.html

49

Templates

• http://www.the-software-tester.com/templates.html

• http://www.softwaretestinghelp.com/resources/

• http://www.gobookee.org/peer-review-checklist-template/

• http://www.mitchlacey.com/resources/in/scrum-tools

50

Q&A