(Maybe We Don’t Have to) Test It
May 25, 2015
(Maybe We Don’t Have to)
Test It
Preparing For Houseguests(A Story)
TheProblem
11
1 1 100
001 00
Tester
11 1
00
01000
10 0
1 0
1 1001 100 00 000 1 11
11 1
Video of arrogant tester.
Routines are safe.
“If it exists, I want to test it”
James Bach
“If it exists, I want to test it. (The only exception is if I have
something more important to do.)”James Bach
11
1 1 100
001 001
1 10
00100
10 0
1 0
1 10
0
1 100 00 000 1 1
11
1
100
100
100
11
1 1 00
01 0010 0
1 101 1001
01
100
1
Patterns
1No repro steps.
Bug106621:DimMedia returns error intermittently.
How can I repro this?
Beats me.
I’ll play around in QA.
Have fun.
Um, no luck yet.
You’re wasting time.
Can’t you debug it or something?
No.
How will we know if it’s fixed before we
release to prod?
We won’t.
2Things that don’t go to
production.
Feature 106218:As a developer, I would like to convert TypeMock unit tests to Microsoft Fakes…
Feature 96120:
Create visualization of
service operation
dependencies from SM
data.
Feature 96190:
Create service
latency reports to
more easily
troubleshoot
problems
3Patches for critical
production problemsthat can’t get worse.
Bug 102927:
Users unable to rework today’s schedule with empty alternate.
Bug 103789:CopyEngine.ApplyVirtualCopyBySchedule causing perpetual LogDataMart timeout.
4Cosmetic changes with
timely test setup.
Feature 103540:
The word “unknown” is
spelled incorrectly on
the alt time constraint
validation message.
Would you bother to test this?
Yes No0
10
20
30
40
5Straight forward
configuration changes.
Feature 106863:
TVE DS: Disable SCTE
events for CNN & HLN
Bug 94337:SolverPlatform jobs throwing error while while trying to serialize parameter.
Maximum number of items that can be serialized or deserialized in an object graph is '65536'
Trust but
verify
6Too technical for a non-
programmer to test.
Feature 106221:
Janus TypeInitializer
should recover if Janus
Admin service is down
upon initial service call
after app pool reset.
7Non-tester on loan.
Feature 100251:Brand change notification cosmetic enhancements.
Feature 95947:
As a module user I
would like the module
search to be resizable so
I can see longer criteria.
Feature 94052:
As a member of the Proof
Group, I would like the
proof categories to be
more intuitive.
8Inadequate test data
or hardware.
Bug 105481:
CDC not properly re-
initializing after re-
initializing replication.
Bug 106271:Database script - users unable to open schedule for TNT, 1/11/2013 due to orphaned comment.
9Bugs not logged by
you.
Techniques
I’m not testing this because...
Are you crazy?
Only a fool would bother
testing this.
I’ll just mark it
“Tested” without doing
much.
An endorsement without careful thought or personal investment in the outcome.
1.Some degree of risk analysis took place.
2.The impact to production would be low.
RR 7/13/12: There is no feasible test
that can be done for this prior to
deployment due to replication not
happening in downstream
environments. The worst case if this
does not work, is CDC will still remain
broken. There is no risk of data loss.
Passing this work item along so it can
be deployed to production.
3 Questions to Start Asking:
1. Does this need to ever be tested?
2. Does it need to be tested by me?
3. Who cares if it doesn’t work?
“NOT testing” may still leave room for testing by
a broader definition
We may be…
• Regression testing/checking.
• Questioning our team.• Learning product
architecture.• Evaluating others’ tests.• Reviewing code.
Instead of saying “NOT testing”,
try saying“testing something more important”.
1. Spend more time on valuable testing.2. Look for “not-test” opportunities. 3. Use my 9 patterns and find your own.4. Introduce a code word.
Video of revised arrogant tester.