Testing as Infrastructure

Created by:

IMPROVING THE EFFECTIVENESS OF TESTING

Tony Marks, Principal Test Manager, YBS Group

Testing as Infrastructure

ABOUT ME

Civilian apprenticeship in Electronics (Ministry of Defence, 1991 – 1995)

In-store Repair Engineer (Mastercare, 1995 – 1998)

Production Test Engineer, Production Test Supervisor, QA Manager, Product Manager (Trilogy Communications, 1998 – 2012)

Test Manager, Principal Test Manager (YBS Group, 2012 – present)

A FEW QUOTES

“If all you have is a hammer, everything looks like a nail.”

The Psychology of Science Abraham Maslow (1966)

“If you always do what you’ve always done, you’ll always get what you’ve always got.” Unknown

“Insanity: doing the same thing over and over again and expecting different results.” Albert Einstein

CHALLENGES WITH PROJECT-FUNDED TESTING

CHALLENGES WITH PROJECT-FUNDED TESTING

Project Managers have ultimate decision-making powers

Typically (even in an agile SDLC), testing happens at the end of the development cycle

Looming deadlines combined with late delivery into Test results in reduced testing time Increased cost (headcount) Reduced confidence (lower coverage)

The testing team are typically perceived as a bottleneck by the rest of the project team

WHAT WOULD NEED TO CHANGE?

WHAT WOULD NEED TO CHANGE?

De-coupling the delivery of testing from the project vehicle would remove Project Managers’ ability to compromise testing effectiveness

An independently funded Testing function with extra-project reporting lines would relieve the Project Managers of the responsibility for testing

BENEFITS TO TESTING

BENEFITS TO TESTING

Independent ability to select and use the most appropriate tools

Flexibility in assigning the most appropriate resources at the appropriate times

Holistic view of the IT landscapeAbility to optimise testing by exploiting synergies between projects

BENEFITS TO TESTING

Enhanced knowledge sharing capability (spreading domain knowledge)

Increased confidence will take the test team to greater heights, accepting greater challenges and leading to higher quality releases into live

BENEFITS TO TESTING

Reflect

BENEFITS TO THE BUSINESS

BENEFITS TO THE BUSINESS

An independent Testing function could engage with projects when most value could be added (as determined by Testing) E.g. Early static testing of functional and

non-functional requirementsReduced overall cost of testing

No longer relying on projects to fund testers when there is little or nothing for them to do

Reduced threat to quality due to conflicting project priorities

BENEFITS TO THE BUSINESS

Consistent standards and processes across projects and the business

Independent assurance of the quality of project deliverables

Testing as an independent entity will get wider recognition and acceptance, enriching the business/test relationship The business will appreciate Testing’s efforts

to act as a gatekeeper on their behalf Increased trust in testing would engender a healthier working culture

BENEFITS TO THE BUSINESS

Reflect

TESTING AS INFRASTRUCTURE

TESTING AS INFRASTRUCTURE

This is a Sync. Pulse Generator (SPG):

It is a critical piece of infrastructure in broadcast facilities

TESTING AS INFRASTRUCTURE

SPG Enables real-time

synchronisation of multiple sources of various formats

Failure can be catastrophic:

Equipment damage Broadcast interruption Negative consumer

experience Maintenance is required:

Routine calibration Repairs Upgrades

Testing Test multiple projects and

changes simultaneously

Failure can be catastrophic:

Regulatory breach Reputational damage Privileged information

loss Negative user

experience Training is required:

New technologies Cross-skilling Knowledge sharing

TESTING AS INFRASTRUCTURE

Reflect

SO HOW DO WE SELL IT?

SO HOW DO WE SELL IT?

Without a Testing function as infrastructure: Simple stand-alone pieces of work would

probably be OK (for a while at least) Complex integrations can be very costly and

time-consuming to support effectively due to the limited scope of any individual project or programme and unclear responsibilities

Headline project costs will be reduced as funding for testing will be excluded

SO HOW DO WE SELL IT?

Project Managers will have one less headache

Testing resources will be more efficiently utilised

An independent Testing function would enable an organisation-wide view of the entire estate, empowering the Testing function to assign resources effectively and assure suitable test coverage

SUMMARY

SUMMARY

Viewing Testing as an essential part of organisational infrastructure provides a different lens

Centralised funding would enable more effective and efficient spending to the benefit of the whole organisation

Quality cannot be compromised by individual projects’ drivers

An independent Testing function would be able to enforce strict quality criteria

THANK YOU!

Questions/discussion