When to Compromise on Testing eurostar 2000 Copenhagen 6 th December 2000 Graham Thomas Testing Manager Wall Street Systems.

When to Compromise on TestingWhen to Compromise on Testing

eurostar 2000 Copenhagen

6th December 2000

Graham ThomasTesting Manager

Wall Street Systems

WALL STREET SYSTEMS 2When to Compromise on Testing

AbstractAbstract

Several times over the last few years working as a tester I have found myself making compromises on the way that I have been testing, and generally felt very uncomfortable about doing so.

Everyone will tell you that compromise in testing is inevitable, but that never makes it any easier. It is never possible to get the perfect mix of resources, skilled testers, equipment to test upon, enough time to plan and prepare for testing, or even to run all of the test scripts, let alone re-test all of the software fixes.

This talk does not offer a silver bullet solution, but will take you through the testing lifecycle, identifying the areas where compromise is most commonly called for, and show you the techniques that I have found successful in managing and controlling that compromise without losing integrity.

And also a few of the pitfalls!

WALL STREET SYSTEMS 3When to Compromise on Testing

Dictionary DefinitionDictionary Definition

Bring oneself under suspicion by indiscrete action

Settlement of dispute by mutual concession

Adjustment of conflicting opinions by modification of each

WALL STREET SYSTEMS 4When to Compromise on Testing

AgendaAgenda

StrategyPlanningResourcesPreparationTrainingMonitoring & ReportingExecution

WALL STREET SYSTEMS 5When to Compromise on Testing

Strategy - CompromisesStrategy - Compromises

We know how to test, just produce a plan.

We are doing rapid development but we want proper testing!

Only one phase of testing actually carried out

Do we even have a testing strategy?

WALL STREET SYSTEMS 6When to Compromise on Testing

Strategy - TipsStrategy - Tips

Brainstorming WorkshopBring all parties togetherModerate the session

The presentation of the material is as important as the strategy document

Keep the strategy document short 15-20 pages, with plenty of diagrams.

WALL STREET SYSTEMS 7When to Compromise on Testing

Strategy - TipsStrategy - Tips

Who, How, Why, What, Where & When.

I keep six honest serving-men (They taught me all I knew); Their names are What and Why and When And How and Where and Who.

Rudyard Kipling -The Elephant's Child

WALL STREET SYSTEMS 8When to Compromise on Testing

Planning - CompromisePlanning - Compromise

You’ve got 10 weeks.We have to go live on Jan 1st

We don’t have planning tools for the testing team!

You do not control the resources needed to carry out testing

WALL STREET SYSTEMS 9When to Compromise on Testing

Planning - TipsPlanning - Tips

Embed the testing plan in the overall project/program plan

Establish the dependency on development deliverables

Use past experience to develop a planning template

Update your plan

WALL STREET SYSTEMS 10When to Compromise on Testing

People - CompromisePeople - Compromise

We have some new recruits that you could have!

The business don’t want to get involved, they’re too busy

All our testers are committed, get some external resource

Organisation does not recognise testing as an IT role

WALL STREET SYSTEMS 11When to Compromise on Testing

My Ideal Testing TeamMy Ideal Testing Team

Works for nothingPrepared to die trying

WALL STREET SYSTEMS 12When to Compromise on Testing

People - TipsPeople - Tips

Mix of business, technical and testing skills

Second from business and ITConsultants can kick start your project

and lead by exampleA well motivated and rewarded team can

achieve the impossibleDon’t overwork the test team!

WALL STREET SYSTEMS 13When to Compromise on Testing

Environment - CompromiseEnvironment - Compromise

You can test in the development environment

Production support has priority over the test environment

We can’t afford new kit, you’ll have to use what we've already got!

WALL STREET SYSTEMS 14When to Compromise on Testing

Environment - TipsEnvironment - Tips

Dedicated EnvironmentAs production like as possibleAgree SLA (Service Level Agreements)

with all systems ownersTake a back-up before you startDocument the configuration

WALL STREET SYSTEMS 15When to Compromise on Testing

Preparation - CompromisePreparation - Compromise

So when do you start the testing?It’s only a small system, what is taking so

long, I can test it in 5 minutes?Why do I need to prioritise this? It’s all

important to me!I am not producing any documentation,

you can have a look at the finished code!We use a copy of production for test data

WALL STREET SYSTEMS 16When to Compromise on Testing

Preparation – TipsPreparation – Tips

Clearly identify how long the preparation activity will take

Prioritise everythingCirculate test planning output for reviewUse a repository for test planningUse the training material in the absence

of systems documentationBack-up your baseline

WALL STREET SYSTEMS 17When to Compromise on Testing

Training - CompromiseTraining - Compromise

We can’t afford training for the testersYou don’t need training in the testing

tool, just use it and see how you get onTest team has no formal training in

structured testing or testing techniques

WALL STREET SYSTEMS 18When to Compromise on Testing

Training - TipsTraining - Tips

Training inTechniques and methodsTools and utilitiesSystem under test

MentoringQuick start consultancy rather

than formal trainingTrain on-site with your own

systems, data and configurationEncourage personal development

WALL STREET SYSTEMS 19When to Compromise on Testing

Execution - CompromiseExecution - Compromise

Development are going to deliver late but we don’t want to change the go-live date!

Not all of the system will be ready in timeProblem reports dismissed by

developmentCarry on testing even when code should

be returned to developmentTesting in parallel with development

WALL STREET SYSTEMS 20When to Compromise on Testing

Execution - TipsExecution - Tips

If you don’t predict your results before you run the test, you are not doing testing

Don’t automate if you don’t have timeReport faults as soon as you find themTight control is required when executing

the regression test packDefer low priority testsBatch up fault fixes

WALL STREET SYSTEMS 21When to Compromise on Testing

Reporting - CompromiseReporting - Compromise

Different view required by management each week

Volume of faults makes it difficult to keep up with reporting schedule

Different view maintained by development team

Reports are several days out of datePressure to embellish the story

WALL STREET SYSTEMS 22When to Compromise on Testing

Reporting - TipsReporting - Tips

Monitor, Measure and the ManageMaintain reporting information in a

repositoryBe flexible when reportingProduce accurate daily status reportsKeep a consistent viewDon’t bombard management with statsBe honest

WALL STREET SYSTEMS 23When to Compromise on Testing

Reporting - TipsReporting - Tips1. Progress against Plans 2. Issues Raised v Closed

3. Test Script Coverage 4. Issue Reporting

WALL STREET SYSTEMS 24When to Compromise on Testing

Summary – What I will doSummary – What I will do

Write a good test strategy presentationEmbed testing in the project planPrepare thoroughlyBack-up my baselineReport faults as soon as I find themPrioritise test executionProvide timely management informationTake it personally when we find a live bug

WALL STREET SYSTEMS 25When to Compromise on Testing

Summary – What I wont doSummary – What I wont do

Just get on with it!Allow testing to be used as development

contingencyCarry out testing without a planBe pressured into curtailing preparationTest on an unrepresentative environmentTest in parallel with developmentPut a gloss on testing progress

WALL STREET SYSTEMS 26When to Compromise on Testing

ConclusionConclusion

You can still test successfully whilst making compromise

A testing methodology is just a means to an end

Your measure of success is the quality of the delivered system, and not adherence to the methodology

Remember both sides give a little

WALL STREET SYSTEMS 27When to Compromise on Testing

Contact DetailsContact Details

Graham Thomas

Testing Manager

Wall Street SystemsWall Street Systems

0044 20 7827 0430

[email protected]