W12 Special Topics
5/1/2013 1:45:00 PM
Presenting Test Results with Clarity
and Confidence
Presented by:
Griffin Jones
Congruent Compliance
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
Griffin Jones
An agile tester, trainer, and coach, Griffin Jones provides consulting on context-driven software testing and regulatory compliance to companies in regulated and unregulated industries. Recently, he was the director of quality and regulatory compliance at iCardiac Technologies which provides core lab services for the pharmaceutical industry to evaluate the safety of their potential new drugs. Griffin was responsible for all matters relating to quality and FDA regulatory compliance including presenting the verification and validation (testing) results to external regulatory auditors. He is currently a host of the Workshop on Regulated Software Testing (WREST). Reach Griffin at [email protected].
4/16/2013
1
Presenting Test Results
with
Clarity and Confidence
April 2013 © 2013 Congruent Compliance LLC 1
Griffin JonesConsultant
Agile / Testing / Regulatory
April 2013 © 2013 Congruent Compliance LLC 2
• Present my current working model of how to:
– clearly and confidently
– prepare and present your testing results
• This presentation covers the Who, What,
When, Where, Why, and How - plus the Traps
– of that model
4/16/2013
2
• Tell a Story
– Audience, Content, Preparation, and Practice
• Intentional Practice
–Clarity• Clearness of expression
–Confidence• Faith in one's abilities
April 2013 © 2013 Congruent Compliance LLC 3
Practice, Practice, Practice
HOW DO YOU GET TO
CARNEGIE HALL?
April 2013 © 2013 Congruent Compliance LLC 4
When are you training your dog?
You are always training your dog.
When are you presenting your
testing results?
You are always presenting
your testing results.
WHEN?
4/16/2013
3
April 2013 © 2013 Congruent Compliance LLC 5
WHO ARE YOU
PRESENTING TO?
Customers
Other Testers
StakeholdersLawyers
Executives
Auditors
Project Managers
Project Team Members
Service
Operations
Development
Partners
Sales & Marketing
April 2013 © 2013 Congruent Compliance LLC 6
WHAT ARE THEIR CONCERNS?
Is project management effective?
What is the status of the project?
Am I getting value?
Will this add value to my business?
Did they follow their
process?
What are the legal risks?
How does this influence the business model?
Am I getting value?
How should I adjust my testing?
Can I support this in the field?
Can I sell this?
How do we
coordinate?
Can I deploy it?
Is the task done-done?
Can I integrate my
stuff with theirs?
How does this
influence my concerns?
4/16/2013
4
April 2013 © 2013 Congruent Compliance LLC 7
WHAT DO
THEY WANT?!
WHEN DO
THEY WANT IT?!
Information!
Now! (you have
two minutes)
• Information based on Observation
– Otherwise it is analysis, induction, and speculation
April 2013 © 2013 Congruent Compliance LLC 8
WHAT KIND OF
INFORMATION?
• Testing is an informational function
– You are paying to Learn something
Adapted from “The Big Book of Perfect Testing” by James Bullock
4/16/2013
5
• Information is valuable if the audience uses it
– Otherwise it is entertainment
• Reduces damaging uncertainty
– (and certainty)
– About the perceived state of
three fundamental questions
April 2013 © 2013 Congruent Compliance LLC 9
WHY DOES IT HAVE
VALUE?
Adapted from “The Big Book of Perfect Testing” by James Bullock
Adapted from “The Testing Team's Motto” by Brian Marick
Adapted from a conversation with Michael Bolton
April 2013 © 2013 Congruent Compliance LLC 10
QUESTION 1:
THE PRODUCT
How good is the
Product?
Adapted from “Rapid Software Test Management” by James Bach
and Michael Bolton
(By “good”, I mean: reduce damaging uncertainty
(and certainty) about the perceived state of …)
4/16/2013
6
April 2013 © 2013 Congruent Compliance LLC 11
QUESTION 2:
THE TESTING
How good is the
Testing?
Adapted from “Rapid Software Test Management” by James Bach and Michael Bolton
(By “good”, I mean: reduce damaging uncertainty
(and certainty) about the perceived state of …)
April 2013 © 2013 Congruent Compliance LLC 12
QUESTION 3A:
HOW GOOD IS YOUR JUDGMENT?
Testing Techniques are Endless
Avoid Ritual
Lich Stefan Schäfer
and Administration
Adapted from “Rapid Software Test Management” by Bach and Bolton
Adapted from “The Big Book of Perfect Testing” by James Bullock
4/16/2013
7
April 2013 © 2013 Congruent Compliance LLC 13
QUESTION 3B:
RITUAL VERSUSADMINISTRATION
The correct technique is the one that best gets you the
information you need to take a better next step
than you would otherwise
Adapted from “Rapid Software Test Management” by Bach and Bolton and “The Big Book of Perfect Testing” by James Bullock
Stuck on a technique
rather than the results,
you have
descended into ritual
Unwilling to discover, embrace,
or invent whatever technique it
takes to find the information
you need, you have
descended into administration
April 2013 © 2013 Congruent Compliance LLC 14
QUESTION 3C:
HOW GOOD IS YOUR CREDIBILITY?
Why should you be happy with the testing
we did and did not do?
More T, Less BSSetup
Bug
Test
Adapted from “Rapid Software Test Management” by Bach and Bolton
Adapted from “More T, Less BS” by Jon Bach
4/16/2013
8
April 2013 © 2013 Congruent Compliance LLC 15
PREPARING TO TELL YOUR
STORY
Always be prepared
15
Model
Competent
Honest
Evidence
Control
Willing
Congruent
April 2013 © 2013 Congruent Compliance LLC 16
CONGRUENT
Equal respect for:
Self, Other, Context
Insides match the outsides
What is the best thing you could say
or do to get the result you want?
Cybernetic Focus
Blame, Placate,
Ultra-Reasonable
Adapted from “More Secrets of Consulting” by Gerald Weinberg and from conversations with James Bullock
4/16/2013
9
April 2013 © 2013 Congruent Compliance LLC 17
THEORY
Honest
Competent
Process
Model
Dishonest
Incompetent
Inadequate
Self-
Incrimination
Experts &
Heroes
Over-
Constrained
• Tests
– How do we react to criticism?
– Are we a Learning Organization? (Fifth Discipline)
April 2013 © 2013 Congruent Compliance LLC 18
HONESTTrust
Sincerity
Integrity
Truthful
Words, Actions, Documents
4/16/2013
10
April 2013 © 2013 Congruent Compliance LLC 19
COMPETENT
Speaks the Language
Understands
Technology and Context
Credible
Capable
Trained in the Industry
• Test
– Are we capable of doing good work?
April 2013 © 2013 Congruent Compliance LLC 20
Complete
PROCESSMODEL
Explainable
Logical
Practical
Reasonable
• Tests
– What problem is the model trying to solve?
– How will the model fail?
4/16/2013
11
April 2013 © 2013 Congruent Compliance LLC 21
PRACTICE
Unwilling
Out-of-
Control
Hand-
Waving
Willing
Under
Control
Evidence
Excessive
Micro-
Management
Obsessive-
Compulsive
• Test
– Do people care enough to provide sufficient resources?
April 2013 © 2013 Congruent Compliance LLC 22
WILLINGFocused
Nurtured
Given AttentionStaffed
Supported
CommittedMotivated
ResourcedPrioritized
4/16/2013
12
• Test
– Are the types and levels of controls appropriate? (prescriptive versus discretionary)
April 2013 © 2013 Congruent Compliance LLC 23
Configuration
CONTROL
Preparationand
Planning
Checking and
Testing
Preventionand
Correction
Review
Accountability
Traceability
April 2013 © 2013 Congruent Compliance LLC 24
EVIDENCE
Reconstructable
Reviewable
Contemporary
Attributable
Human Readable
RetrievableObjective
Clear
• Test
– How does specific evidence meet the criteria?
4/16/2013
13
April 2013 © 2013 Congruent Compliance LLC 25
PRESENTATION TRAPS
Too formal
Too much
preparation required
Too little content
Not knowing
the audience
Over or
under-telling stories
Not listening
and engaging
Presentation format and
content abdication
Meaning without Data, or
Data without Meaning
Not knowing your
purpose and role
Lack of practice
Clarity
Confidence
• Practice, Practice, Practice
• You are always presenting your results
• Know the concerns of your different
stakeholders
• Stakeholders want fast information, or fast
learning
– Based on observation
– Used to inform a decision
April 2013 © 2013 Congruent Compliance LLC 26
SUMMARYA …
4/16/2013
14
April 2013 © 2013 Congruent Compliance LLC 27
… SUMMARY B
• Information is valuable because it reduces
damaging uncertainty and certainty
– about the product, testing, and your judgment
and credibility
• Prepare by monitoring whether you are
Congruent
• While focusing on the six qualities of: Honest,
Competent, Process Model, Willing, Control,
and Evidence
April 2013 © 2013 Congruent Compliance LLC 28
BIG POINTSRelax - You know what you are doing
Keep it SimpleYou Practice this
all the time
Clarity
Confidence
4/16/2013
15
• “Rapid Software Test Management” by James Bach
and Michael Bolton
• “More T, Less BS” by Jon Bach
• “The Big Book of How to Do Perfect Testing” by
James Bullock
• “Surviving an FDA Audit” by Griffin Jones
• “The Testing Team’s Motto” by Brian Marick
• “More Secrets of Consulting” by Gerald M Weinberg
April 2013 © 2013 Congruent Compliance LLC 29
REFERENCES
April 2013 © 2013 Congruent Compliance LLC 30
QUESTIONS
4/16/2013
16
Thank you for attending this session.
Please fill out the evaluation form.
Griffin Jones
April 2013 © 2013 Congruent Compliance LLC 31