AD42%!Reality%Driven!Testing!in!Agile Projects! · Instructor Introduction Presented By Robert Sabourin robsab@gmail.com © 2019 Robert Sabourin RDT v1.0 2

     AD42  Agile  Testing  &  Test  Automation  3:15  PM            

AD42  -­‐  Reality-­‐Driven  Testing  in  Agile  Projects  

 Presented  by:    

Robert  Sabourin      AmiBug.Com,  Inc.  

 Brought  to  you  by:    

   888-­‐-­‐-­‐268-­‐-­‐-­‐8770  ·∙·∙  904-­‐-­‐-­‐278-­‐-­‐-­‐0524  -­‐  info@techwell.com  -­‐  https://agiledevopswest.techwell.com/    

Robert  Sabourin    Rob  Sabourin  has  more  than  thirty-­‐five  years  of  management  experience  leading  teams  of  software  development  professionals.  A  highly-­‐respected  member  of  the  software  engineering  community,  Rob  has  managed,  trained,  mentored,  and  coached  hundreds  of  top  professionals  in  the  field.  He  frequently  speaks  at  conferences  and  writes  on  software  engineering,  SQA,  testing,  management,  and  internationalization.  Rob  authored  I  am  a  Bug!,  the  popular  software  testing  children's  book;  works  as  an  adjunct  professor  of  software  engineering  at  McGill  University;  and  serves  as  the  principal  consultant  (and  president/janitor)  of  AmiBug.Com,  Inc.  Contact  Rob  at  rsabourin@amibug.com.    

REALITY DRIVEN TESTING IN AGILE PROJECTS

Instructor Introduction

Presented By

Robert Sabourin

robsab@gmail.com

2© 2019 Robert Sabourin RDT v1.0

© 2019 Robert Sabourin RDT v1.0 3

Reality Driven Testing

• Robert Sabourin ,

Software Evangelist

• President

• AmiBug.Com Inc.

• Montreal, Quebec,

Canada

• robsab@gmail.com

1SOME PHILOSOPHY

© 2019 Robert Sabourin RDT v1.0 5

Reality Driven Testing

• Pain points?

– What hurts?

– How Much?

© 2019 Robert Sabourin RDT v1.0 6

Fundamental Question

• How do you know when you are finished?

© 2019 Robert Sabourin RDT v1.0 7

Reality Driven Testing

© 2019 Robert Sabourin RDT v1.0 8

Reality Driven Testing

Phil Crosby

• “Quality is defined as conformance to

requirements”

• “Quality is not a measure of

GOODNESS”

– Phil B. Crosby, Quality is Free

© 2019 Robert Sabourin RDT v1.0 9

Reality Driven Testing

“Quality is fitness for use”

Joseph Juran

Quality Control Handbook

© 2019 Robert Sabourin RDT v1.0 10

Reality Driven Testing

Gerald M. Weinberg

“Quality is value to some person”

Exploring Requirements Quality Before Design

Dorset House

© 2019 Robert Sabourin RDT v1.0 11

Reality Driven Testing

Conforming to requirements

Suiting purpose

Delivering value to stakeholders

© 2019 Robert Sabourin RDT v1.0 12

Reality Driven Testing

Circa 25 BC

© 2019 Robert Sabourin RDT v1.0 13

Reality Driven Testing

Circa 1979 AD

© 2019 Robert Sabourin RDT v1.0 14

Reality Driven Testing

Edsger W. Dijkstra

• “Program testing can be used to show

the presence of bugs, but never to show

their absence”

© 2019 Robert Sabourin RDT v1.0 15

Reality Driven Testing

Real changes

Real usage

Real world

Value Sync

© 2019 Robert Sabourin

RDT v1.0

Slide 16

© 2019 Robert Sabourin RDT v1.0 17

Charter Types

Usage Scenarios

– Identify classes of

users

– Identify how users

will use system

– Describe scenarios

– Use Story board or

similar approaches

– Identify variations

Usage Scenarios

18© 2019 Robert Sabourin RDT v1.0

Pareto Combinations

19© 2019 Robert Sabourin RDT v1.0

0

2

4

6

8

10

12

14

16

1 17 33 49 65 81 97 113 129 145 161 177 193 209 225 241

Count

Count

Usage Scenarios

20© 2019 Robert Sabourin RDT v1.0

Usage Scenarios

21© 2019 Robert Sabourin RDT v1.0

Usage Scenarios

22© 2019 Robert Sabourin RDT v1.0

Usage Scenarios

23© 2019 Robert Sabourin RDT v1.0

Usage Scenarios

24© 2019 Robert Sabourin RDT v1.0

Usage Scenarios

25© 2019 Robert Sabourin RDT v1.0

Usage Scenarios

26© 2019 Robert Sabourin RDT v1.0

Usage Scenarios

27© 2019 Robert Sabourin RDT v1.0

Value Sync

© 2019 Robert Sabourin

RDT v1.0

Slide 28

© 2019 Robert Sabourin RDT v1.0 29

Reality Driven Testing

Quiz

• Application screens

are selected with

three controls:

(a) has 5 options

(b) has 6 options

(c) has 2 options

• How many screens can a user choose?

© 2019 Robert Sabourin RDT v1.0 30

Reality Driven Testing

Quiz

• Total Combinations

= 6 x 5 x 2 = 60

• To exercise each

combination once a

total of 60 tests

would be required.

© 2019 Robert Sabourin RDT v1.0 31

Reality Driven Testing

Quiz

• How many tests

would be required to

exercise all possible

screens in every

possible order?.

© 2019 Robert Sabourin RDT v1.0 32

Reality Driven Testing

Quiz

• To exercise all screens in every

possible order would require

60! Test cases

60! = 60 x 59 x 58 x ... 3 x 2 x 1

60! ≈ 8.32 x 10**81

© 2019 Robert Sabourin RDT v1.0 33

Reality Driven Testing

Quiz

From 7.0 × 10**79

To 1.5 × 10**82

How many atoms are in

the observable

universe?

© 2019 Robert Sabourin RDT v1.0 34

Charter Types

Sequences

– Explore paths

– Vary

– Operation order

– Sequences

– Valid

– Invalid

– Multiple

– Concurrent

Control Flow Testing

35© 2019 Robert Sabourin RDT v1.0

Control Flow Testing

36© 2019 Robert Sabourin RDT v1.0

Control Flow Testing

37© 2019 Robert Sabourin RDT v1.0

Control Flow Diagram to Identify Basis Paths

Basis Paths

p01 Start Dispatch L M N P Q R

p02 Start Dispatch X Y AA AB R

p03 Start Dispatch AC AF AJ AI AK R

p04 Start Dispatch AD AE AF AJ AI AK R

p05 Start Dispatch O P Q R

p06 Start Dispatch K P Q R

p07 Start Dispatch I J C D H P Q R

p08 Start Dispatch A B C D H P Q R

p09 Start Dispatch L M N P S T Q R

p10 Start Dispatch L M N P S T U R

p11 Start Dispatch L M N P S T U V W R

p12 Start Dispatch X Z AA AB R

p13 Start Dispatch X Z AA AM R

p14 Start Dispatch X Z AA AM AN AP AQ R

p15 Start Dispatch X Z AA AM AN AO AQ R

p16 Start Dispatch AC AF AG R

p17 Start Dispatch AC AF AH R

p18 Start Dispatch AC AF AJ AL AM R

p19 Start Dispatch AC AF AJ AL R

p20 Start Dispatch I J C D E F R

p21 Start Dispatch I J C D E G R

p22 Start Dispatch A B R

Nodes 45

Edges 65

Complexity 22 E-N+2

Value Sync

© 2019 Robert Sabourin

RDT v1.0

Slide 38

Charter Types

Failure ModesW

rap

-O-M

ati

c

© 2019 Robert Sabourin RDT v1.0 39

Charter Types

Failure ModesChocolate Receiver

Chocolate Weight Sensor

Paper Receiver

Box Receiver

Ribbon Receiver

Manifest Entry Console

Material Data Manager

Inbound Material Error Processor

Wrapping Engine

Ribbon Engine

Boxing EngineOutbound Boxes

Manager

Rejected Chocolate Manager

Transaction Journal Manager

Report Generator

Manifest Data Manager

Inbound conveyor belts

Inbound paper trays

Inbound box racks

Inbound ribbon spools

Wrap O Matic Ignition

Report Printer

Wrap O Matic Controller

Outbound Boxes

Outbound Rejected Chocolates

Operator Control Entry

Wrap O Matic Status Indicators

Wrap O Matic Process Block Diagram

Backup

Blo

ck D

iag

ram

© 2019 Robert Sabourin RDT v1.0 40

Charter Types

Failure Modes

For each object I can ask the question:

What if the object fails during a transaction?

What if the object is not visible?

What if the object is busy?

© 2019 Robert Sabourin RDT v1.0 41

Failure Mode Analysis

42© 2019 Robert Sabourin RDT v1.0

Failure Mode Analysis

43© 2019 Robert Sabourin RDT v1.0

Testing Ideas

• Investigative

approaches

– We become truffle

snorting pigs and try to

find useful information in

all evidence we discover

– We can even get good

ideas from out of date

sources

Cap

ture

te

sti

ng

id

eas

© 2019 Robert Sabourin RDT v1.0 Slide 44

Testing Ideas

Cap

ture

te

sti

ng

id

eas

Who is paying for the software?

Who is supposed to use the software?

What problem is the software trying to solve?

Does other software have to run before after or during operation?

Will other unrelated software be running at the same time?

© 2019 Robert Sabourin RDT v1.0 Slide 45

Testing Ideas

Cap

ture

te

sti

ng

id

eas

Will data be shared with other software?

Will processing resources be shared with other software?

What are the users going to do with the software?

Are the users familiar with the problem?

Are the users familiar with the solution?

© 2019 Robert Sabourin RDT v1.0 Slide 46

Testing Ideas

Cap

ture

te

sti

ng

id

eas

Are there novice users?

Are there expert users?

Are there typical users?

Are there different categories of users doing different tasks?

Who will install the software?

© 2019 Robert Sabourin RDT v1.0 Slide 47

Testing Ideas

Cap

ture

te

sti

ng

id

eas

Who will configure the software?

Who will manage the software?

Who will manage the data?

Where will data come from?

Does the software contain elements we do not need?

© 2019 Robert Sabourin RDT v1.0 Slide 48

Testing Ideas

Cap

ture

te

sti

ng

id

eas

Are there any non-functional needs?

Does the software have any environmental concerns?

What about operator locale?

Does this software replace existing software?

Does the software replace existing hardware?

© 2019 Robert Sabourin RDT v1.0 Slide 49

Testing Ideas

Cap

ture

te

sti

ng

id

eas

Does the software automate previously manual work?

Is the software being reused?

Is the software being repurposed?

Is the software being customized?

Did the software purchaser understand risk?

© 2019 Robert Sabourin RDT v1.0 Slide 50

Thank You

• Questions?

© 2019 Robert Sabourin RDT v1.0 51