Bug deBug Chennai 2012 Talk - Driving innovation using pattern based thinking by Vipul Kocher

Driving Innovation Using Pattern Based Thinking

1

From India With Love

2

Agenda

Patterns in real world Design Patterns and Test Patterns Usefulness of patterns in testing Patterns in testing

Data Patterns Test Patterns Defect Patterns Test management Patterns People management Patterns Control Patterns

What we do not know today What we might need to create – pattern language

3

What Do You Mean You Have Seen This Before?

4

What Do You See Here?What Patterns Do You See Here?

5

Did You See This?

The sum of the numbers in any row is equal to 2n

Source - http://ptri1.tripod.com/

6

Learning the Language

How do you learn your native language?

How do you learn your second language?

7

*Brain – A Pattern Machine The brain is a pattern-mad supposing

machine. Given just a little stimuli, it divines the probable. …it recognizes familiar patterns and acts with conviction.

As much of a confection as our mental maps are, they (Patterns) allow us to speculate, rehearse and make plans.

*http://www.nytimes.com/2004/06/15/science/essay-i-sing-the-body-s-pattern-recognition-machine.html

8

Pattern

*A form or model proposed for imitation

*Something designed or used as a model for making things

 Design pattern - general reusable solution to a commonly occurring design problem

From MW Dictionary* 9

Patterns in Testing-Possibilities

I predict that one of the future breakthrough in testing will come through pattern recognition

Example Patterns of defects arising out of

patterns of errors in code/design/requirements

Patterns of project failures arising out of data collected during project execution

10

Why Identify Patterns?

Patterns exists everywhere - in defects, code, design, processes

Pattern identification helps predict the next set of events in the chain enabling you to do risk mitigation

Patterns enable you to distinguish the woods from the trees

11

Patterns in testing

Data Patterns Test Patterns Defect Patterns Test management Patterns People management Patterns Control Patterns

12

Data Patterns

Ross Collard listed a number of data patterns for functional and performance testing - Routine Live Data Baseline Batch Volume or Parallel Benchmark Pristine Truncated Minimally Redundant etc.

13

Simple Testing Pattern (Observed)

People often work on one side of boundary while testing

Make them work on both sides/all sides of boundary

Example – Triangle Problem negative test cases - no

value, 1 value, 2 values, all 3 values what about the other side of the

boundary - more number of values?

14

Discovering Test Design Techniques

Patterns can help uncover new test design techniques OR new ways to look at existing test design techniques Apply test design techniques to create tests Monitor the bugs arising OUT(SIDE) of tests Establish the bugs that could have been

found using existing techniques Defects not “findable” using techniques

should be analyzed for patterns and a technique using which these could be found

15

Example of Discovered Technique

Some patterns in type of bugs not found using tests created using techniques Interaction of application with the operating

system Reason – Design specifications hardly go to that

level of detail and system specifications never Defects related to interdependency of features

and code structures causing one portion of code to be bound to another piece of code Reason – Code dependencies are either not called

out explicitly, born in either assumptions or ignorance (most of the time)

16

Unified Test Design Technique

A model based technique

Resembles state diagrams

Entity under test

User Inputs

System Inputs

Input Data

Input Actors

Ass

oci

atio

ns

Co

nta

ins

Output Data

Output Actions

On otherelements/

entities

On OS - Disk- File- Process- Screen IO- System Messages

Transient /Stable State

Parameters & their attributes

Parameters

State of Parameters

En

viro

nm

en

t (E

xt.

Sta

te)

St a

te (

Inte

rna

l Sta

t e)

17

Defects, Patterns and You

How many bugs do you find every month?

How many of them are type – UI, Feature, cross-feature, database, security…

Can you sub-classify them? Do you know what are other testers

finding? Do you see any patterns in

What type of bugs you find What types of bugs you don’t find?

18

Design, Code and Requirements

If you find predictable bugs (based on your past experience and pattern recognition) can there be pattern recognition for code? Design? Requirements?

19

Bug Isolator Patterns

Are there testers who do good bug isolation?

Are there those who do not? How do they fail? What things they

do not do? What patterns exist? If I know those patterns can I

improve their skills?

20

Test Management Patterns

Example Project Failure pattern –

Communication/delivery/no-interested-client-stakeholder

Automation Outsourcing Failure – Lack of ~effort appreciation/well-written tests/domain knowledge

21

Project Diary as a Tool

What pattern do you see here? This pattern emerges from long term view - The four

entries here are from a list of 133 entries between Jan and April 2001. How is pattern recognition different from Root cause analysis?

Date Action Issue24-Jan-

01Clear work division in overlapping areas and ensuring communication between the overlapping/concerned people.

Performance test cases being written by Joe and Jane. Test were duplicated. Some missing test cases because every body was not aware hence did not review/contribute to the tests.

28-Jan-01

Knowledge of the system (WES, WAS performance, tuning, deployment) should be available to the team.

Some members of the team did not know what they had to do, others did not have required technical knowledge

14-Feb-01

Communication of the documents, Agenda, Minutes to the testing team should be formal, via mail.

The release meeting with developers happened without a copy of test plan (esp. dates) being available.

9-Mar-01

User Guides esp. “how to get started” should always accompany a build

Swami sent a mail asking how to use the ABC app. We had to write some steps and send him the document.

22

People Management Patterns

Example – Patterns while Interviewing assessing people, forming impressions,

validating, creating and refining patterns Remember - you have not discovered

the last pattern hence you may be wrong

Keep updating your impression of patterns and be ready for the worst

23

People Management Patterns

Example – “Only a Guest” pattern when people (are about to) leave the

organization without information to the management there are some behavioral patterns that manifest themselves.

Indifference, extra-hard work, sudden forgiveness and not picking up of battles, unplanned absenteeism, putting all paper-work in place etc.

24

Applying Pattern Thinking

Patterns for Predicting Inventing Improving Preventing …

25

Possibilities

Unified Modeling Language – Unified Testing Language

No guarantee of success and no predictions of failure

26

How to Identify Patterns

Applying patterns requires Ignoring the small and looking at the

large Large amount of data (usually) Reduction in the number of variables Elimination of outliers Pattern of outliers Variations and taking cognizance of

those Variations and eliminating them

27

What Do You See Here?

http://www.rense.com/general86/stun.htm28

And Here?

http://www.rense.com/general86/stun.htm

29

Careful…

“Homo sapiens is about pattern recognition, he says. Both a gift and a trap.” - William Gibson – Pattern Recognition

30

Some Interesting Links

http://www.exampler.com/testing-com/test-patterns/

http://c2.com/cgi-bin/wiki?TestingPatterns http://www.testingreflections.com/node/view/9

31

Questions?

Thank you everybody <<meta-pattern – help>> for images, text and ideas

32