Session 2Session 2 11
Testing – Why We Do Testing – Why We Do It BadlyIt Badly
Alan E. Devney, Jr. PMPAlan E. Devney, Jr. PMP
FusionSoft LLCFusionSoft LLC
Session 2Session 2 22
Quickly About me!Quickly About me!
Senior Project Manager – Just like Senior Project Manager – Just like most of you!most of you!
Notable customers: Walt Disney Notable customers: Walt Disney World Company, Sega, Duke World Company, Sega, Duke Energy.Energy.
Currently a Project Manager at Currently a Project Manager at FusionSoft LLC.FusionSoft LLC.
Session 2Session 2 33
If I do not test, I can If I do not test, I can still believe that it still believe that it works!works!
Session 2Session 2 44
Project Management Project Management ProverbsProverbs
Project Management is just Project Management is just management with a fancy name.management with a fancy name.
We do not know what the We do not know what the expectations are but we are expectations are but we are pretty sure you are not meeting pretty sure you are not meeting them.them.
Session 2Session 2 55
The Final FrontierThe Final Frontier
Testing is the Last Element of a Testing is the Last Element of a Project’s Success that is still being Project’s Success that is still being defined at its core. defined at its core.
Terms are inconsistent in the Terms are inconsistent in the industry. Is it a tomato or a tamato?industry. Is it a tomato or a tamato?
A large portion of Project A large portion of Project Professionals are still learning how Professionals are still learning how and what it means to test effectively.and what it means to test effectively.
Session 2Session 2 66
PeoplePeople
Session 2Session 2 77
Sponsor SupportSponsor Support
We spent so much time on We spent so much time on requirements, design, use cases, requirements, design, use cases, test cases, pillow cases…why do test cases, pillow cases…why do we need to do so much testing?we need to do so much testing?
‘‘We told you what we wanted, We told you what we wanted, didn’t you use our requirements didn’t you use our requirements to build what we asked?’to build what we asked?’
Session 2Session 2 88
IT Management IT Management SupportSupport We have not done it for the last 20 We have not done it for the last 20
years why start now?years why start now? Lack of understanding of new Lack of understanding of new
technology.technology. Production Support can clean it up.Production Support can clean it up. We must be on time, on scope, and We must be on time, on scope, and
on budget for me to get my bonus.on budget for me to get my bonus.
Session 2Session 2 99
Lack of Core Business Lack of Core Business CommitmentCommitment Business Users have a full time job Business Users have a full time job
already.already. Business Users do not see the need for Business Users do not see the need for
the new system.the new system. Business Users do not want to test at a Business Users do not want to test at a
detailed level.detailed level. Implementing new systems is not their Implementing new systems is not their
job!job! I am not really sure what we do here!I am not really sure what we do here!
Session 2Session 2 1010
ProcessProcess
Session 2Session 2 1111
Declining Time to TestDeclining Time to Test
Where are the areas that can be Where are the areas that can be reduced with the least amount of Risk?reduced with the least amount of Risk?
Communication of overall Risk Communication of overall Risk associated with cutting testing corners.associated with cutting testing corners.
Communication with Production Support Communication with Production Support Management.Management.
Ultimately it is a Business decision to go Ultimately it is a Business decision to go live. live.
Session 2Session 2 1212
Lack of Detailed Lack of Detailed RequirementsRequirements If we can not say what we are If we can not say what we are
building we can not clearly define building we can not clearly define how to test it.how to test it.
This allows us to test and believe we This allows us to test and believe we are OK because we did not know are OK because we did not know what we were building in the first what we were building in the first place.place.
The most quoted reason for project The most quoted reason for project failure. failure.
Session 2Session 2 1313
Lack of Proper ToolsLack of Proper Tools
Testing 100,000 lines of test scripts Testing 100,000 lines of test scripts can not be managed using Excel.can not be managed using Excel.
The right tool for the right job. Small The right tool for the right job. Small projects small process, larger projects small process, larger projects require greater addition to projects require greater addition to detail.detail.
Suggestion: Mercury Quality CenterSuggestion: Mercury Quality Center
Session 2Session 2 1414
Testing Role DefinitionTesting Role Definition
Who is doing what?Who is doing what? Are they the right person, Are they the right person,
discipline for the job?discipline for the job? How does my business align with How does my business align with
your business?your business? Estimating level of effort. Can one Estimating level of effort. Can one
person run 30,000 lines to test person run 30,000 lines to test scripts in a month?scripts in a month?
Session 2Session 2 1515
Lack of MeasurementsLack of Measurements Where are we anyway?Where are we anyway? How will senior managers know where the How will senior managers know where the
project stands?project stands? How will the PM know where the project How will the PM know where the project
stands?stands? Does not need to be complex just clear.Does not need to be complex just clear. ExamplesExamples
– What is a Critical Defect?What is a Critical Defect?– Number of ScriptsNumber of Scripts– Estimated Time to Test Estimated Time to Test – Estimated DollarsEstimated Dollars
Session 2Session 2 1616
Lack of Defect Lack of Defect Management ProcessManagement Process How are defects managed?How are defects managed? Does the defect need to be Does the defect need to be
addressed?addressed? Who corrects the defect?Who corrects the defect? Who re-tests the defect?Who re-tests the defect? Who closes the defect?Who closes the defect? SOX?SOX?
Session 2Session 2 1717
Doing it Doing it Right!Right!
Session 2Session 2 1818
Plan!!!Plan!!!
Session 2Session 2 1919
How to Plan Like a Tester
• Mirror image of production environment hardware, protocols, configuration, etc Mirror image of production environment hardware, protocols, configuration, etc.
• Existing & New Infrastructure Requirements• Other components that need to run concurrently
during test• Business Scenarios• Server profiles• Workloads• Data Population• Metrics -Decisions about what is acceptable or
“good enough”
Session 2Session 2 2020
360 Degree Testing360 Degree Testing
TechnologyTechnology Business ProcessBusiness Process ApplicationApplication DataData LocationLocation OrganizationOrganization
Computer Science Corporation – Six Domains of ChangeComputer Science Corporation – Six Domains of Change
Session 2Session 2 2121
What Makes Good What Makes Good TestingTesting Where does Testing Begin?Where does Testing Begin?
– Scope DefinitionScope Definition– Business Process DefinitionBusiness Process Definition– Requirements DefinitionRequirements Definition– DesignDesign– Test Cases/ScriptsTest Cases/Scripts– Acceptance Criteria- Get it in Acceptance Criteria- Get it in
Writing!Writing!
Session 2Session 2 2222
What solid testing is…
Getting acceptable “good enough" metrics (measurements) for:– Correctness of function: is the basic
workflow working and flowing?– Data integrity: Is the right data where it
should be?– User Interface: Security, Likeable, Good
error messages– Documentation: End user and administrator– Environmental Factors: Network traffic,
server busy times, patches– Usage (load): When and how much? Where?
Session 2Session 2 2323
Acceptance Criteria Acceptance Criteria define, as
briefly as possible, the rules the client will use to determine whether the work has been successfully completed.
Know what “Done” looks like specifically. This ties directly to “Measurements.”
Session 2Session 2 2424
UAT Facilitated UAT Facilitated TestingTesting Super Users working directly with Super Users working directly with
Business Testers.Business Testers. Mixes Training and TestingMixes Training and Testing Keeps testing moving! Keeps testing moving!
Session 2Session 2 2525
Good Testing Practice Good Testing Practice Should:Should: Ensure that tests are being planned Ensure that tests are being planned
to cover all relevant aspects of an to cover all relevant aspects of an equipment or computer system, and equipment or computer system, and that the test is being executed and that the test is being executed and documented well enough to enable documented well enough to enable tracing of the test, the test results, tracing of the test, the test results, the handling of deviations and the the handling of deviations and the responsible persons for each responsible persons for each activity. activity. Source: The Good Automated Manufacturing Practice Forum (GAMP 3)
Session 2Session 2 2626
Effective testing is Effective testing is essential for achieving essential for achieving successful business successful business outcomes.outcomes.
Session 2Session 2 2727
QuestionsQuestions