System Testing and Quality: How Real Partnerships Deliver Accelerated Time-to-Market & Increased Revenue David Gehringer VP Marketing, Fanfare.

System Testing and Quality: How Real Partnerships Deliver Accelerated Time-to-Market & Increased Revenue

David GehringerVP Marketing, Fanfare

About Fanfare• Fanfare provides software solutions to 70 global high-tech

equipment manufacturers and service providers that simplify and accelerate system and device testing.

• Our technology enables QA teams and developers to build and run automated tests, measure pass-fail criteria and automatically generate documentation that can be easily shared.

We help accelerate time to market while increasing product quality.

Quality Costs

• Quality may not be job 1, but it should be• Poor quality….

– Costs in development cycles– Lengthens testing – Delays releases– Create customer support costs– Deteriorates brand equity– Cost 100x to fix once in the field

Defect Case Study• Global leading Carrier and Manufacturer

• Realize these costs are for each company• Deep breath…

– None of these costs resolve the defects

Action Time

Communication about defect(Time Zones / Language barriers)

2 Days

Average # of defects discussed(mostly bugs but some are not)

400

Time reporting defects 800 Man Days (~4 man years)

Cost to discuss defects / release ~$500k

Product Release Schedule

Device Testing Device Testing

Service ProviderDevice Testing & Feature Validation

Service ProviderDevice Testing & Feature Validation

Interoperability and System Testing

Interoperability and System Testing

Dep

loym

ent

Dep

loym

ent

Months to Market from first production prototype

Quality is Really Simple

Developers

CustomerAdvocacy

Support

Automation

ServiceProvider

•Feature testing •Regression Testing•Abstraction•Response map ownership•Testbed maintenance•Only Legacy on tcl/perl

Testers

•System Testing

Personal Regression

LegendTC – Test caseTB – TestbedTR – Test ReportRM – Response MapCR – Capture ReportProc - Procedures

•System Testing•Interoperability testing•Device testing•Regression Systems

•Unit Testing•Build Regression

Quality Workflow

• Testing assets should flow from the manufacturer to the service provider

– The tests should have already passed– Assets reduce the work of the service provider– Speeds identification of defects– Triage defect causes quickly

What Assets to Share

• Test Cases• Test Reports• Setup & tear down procedures• Test bed / topology files• Abstraction

– Output, such as device response, GUI elements, Data sets, etc..

– Device centric variables

What is Failing

• In-house systems– Information and assets hard to move and share – Sapping the best talent to maintain– Creates a center of the universe problem

• Scripting– Similar to coding and is hard to understand– Small talent pool – Brittle and costly to maintain

Testing Relationship

Value Chain of Quality

Quality is Simple

Developers

CustomerAdvocacy

Support

Automation

ServiceProvider

•Feature testing •Regression Testing•Abstraction•Response map ownership•Testbed maintenance•Only Legacy on tcl/perl

Testers

•Version Control•Response Maps•Procedures

•System Testing

TC

RMProcs

RM, Procs RM, Procs

TC TR

CR TRTC

TC TR

TC TR

TC TRTBProcs

Personal Regression

LocalRM, TB, Procs

LegendTC – Test caseTB – TestbedTR – Test ReportRM – Response MapCR – Capture ReportProc - Procedures

•System Testing•Interoperability testing•Device testing•Regression Systems

•Unit Testing•Build Regression

Asset Repository

Asset Repository

Quality Business Equation

It is in the financial best interest of both manufactures and service provider to work as partners for quality improvement.

• Quality Issues– Delay sales, delay deployment, and raise support– Reduce both the top line and bottom line – Customer loss and brand erosion

Service Provider Trends

– Business advantages to NEMs who provide assets

– Better relationships drive ongoing costs down

– RFPs that require test assets from NEMs

The Good Samaritan

It is not greening the planet, but high quality makes good business sense.

Those who figure this out will have significant market advantage.

Q & A

David Gehringer. [email protected]