Offshore Testing Pitfalls and Strategies to Help Succeed Tracey Clark Michael Cookson SWOSQG Meeting October 2nd 2012.

Offshore Testing

Pitfalls and Strategies to HelpSucceed

Tracey ClarkMichael Cookson

SWOSQG MeetingOctober 2nd 2012

Agenda

• Why offshore testing?• Choosing the team• Four areas:

– Getting Started– Executing the project– Finalizing the project– Lessons Learned

• Final Q&A– If at any time, you want to ask a question or

debate a point, shout it out

Common pitfalls

What do you think some of the common pitfalls of

offshore testing?

Offshore Testing

Why Offshore?

Offshore Testing

• Definition– Using a test team located in another

country to perform some or all aspects of a projects testing

– May or may not be part of your company

– May or may not be a one time project

Offshore vs. Nearshore

• Nearshore is usually considered same geographical region– Waterloo -> Newfoundland

• Offshore usually different geo– Waterloo – India / China

Offshore vs. Out-sourcing

• Offshore – is the act of moving business functions, that is in house and a part of your region, to another lower cost region

• Outsourcing – loosely defined, but basically means use of a third party company outside your organization to perform a function for your company

Out-sourcing vs. In-sourcing

• Out-sourcing – contracting of an entire function to another company

• In-sourcing – moving an entire function to another area of the company

• Out-sourcing and in-sourcing can involve offshore, nearshore or resources in same locale as you

Out-Sourcing In-Sourcing

Offshore

Having a 3rd party companytake over your wholeIT Support function andrunning it with resourcesin Brazil

Moving your IT supportfunction to in houseresources located inBrazil

Nearshore

Having a 3rd party companytake over your whole ITsupport function and running it with resources In the same city or region

Moving your IT Supportfunction to in house resources located inNewfoundland

Offshore Testing

Offshore Testing

• Why is your company doing this?– Critical to understand this

• A one time project to deal with a resource shortage• A cost saving, short term solution• A long term strategic initiative

– Where does your project relate to the goals above• One of the first to be performing this• Part of a well-established process

Offshore Testing

Choosing the Team

Offshore Testing

• Choosing the team / company you will be working with– Many pitfalls here– How do you verify capabilities?– How can you verify claims made?– The path you choose will depend on why

you are doing this in the first place

Offshore Testing

• Selecting Your Company– If your company has been doing this for

a while, is there a knowledge base you can tap into?• Has this company been used before by your

company?• References – any other company you can go

to?– What is the turnover rate of this

company in the last year?

Offshore Testing

• If this is strategic and long term, do things like any other new strategy– Start small – pick a pilot project that:

• Is not a ‘break the company’ project• Does not have extremely short deadlines• Does not have many other high-risk items

Offshore Testing

Getting Started

Offshore Testing

• Getting started - Questions to ask– How do you get the offshore team up to

speed on:• How the project is to be run• Documentation standards and storage• Reporting standards • Other internal standards and procedures

that are critical to your project success

Offshore Testing

• Getting Started– How many people are on the offshore

team and who are they?• Need to set up usernames, security

– Do they have people who can handle hardware / installation and network issues?• If you ship equipment, is there someone

there who can make sure the set up works?• Is that part of the contract?

Offshore Testing

• Getting Started– Licensing issues

• Many vendors have geo specific licenses – if you give access are you in violation

– Import / Export regulation• Software and hardware (if it needs to be shipped) are

subject to import / export regulations for both countries. Do you understand the implications if any?

– Intellectual property• What can you share and what can not?

Offshore Testing

• Getting Started– Where will the testing environments be?

• Do you need to ship equipments – are there specific hardware / operating system / software

– What internal systems will the offshore team need access to? • Mail; defect tracking; automated script

repositories; documentation sites• 24 x 7 IT / Support desk?

Offshore Testing

• Getting Started– Is there training on the applications / tools

required• Are all your trainers local and do you need to send

them out? Was that budgeted for?– Network connectivity

• Firewall and bandwidth issues (performance)• Especially if offshore team needs to access tools

remotely– System Availability

• Is there a regular maintenance window for the systems you are using that will conflict with the hours of operation for the offshore team

Offshore Testing

• Communication Strategies– Webex– Video conferencing– Conference calls – can you dial

internationally to all locales– Messaging – MSN, gmail, Sametime …– Project repositories / team rooms

Offshore Testing

Executing the Project

Offshore Testing

• Executing the Project– Ensure you understand the contract

• What is explicitly included / excluded?

– Make sure everyone is on the same page before the project actually begins regarding these expectations

Offshore Testing

• Executing the Project– Most companies have some governance

model for running projects• Does yours’ include sections on dealing with

offshore?– Yes – what do you need to do differently?– No – what needs to be added?

– Communication• Keeping the team in the loop on what has

been done, what the issues are and what the focus is

Offshore Testing

• Executing the Project– Do you need an onshore resource?

• Someone who represents the offshore team but is situated locally. Understand labour laws and what is allowed. This person is usually part of the offshore company, not a resource on your team.

– Staggering start times for all teams– Measurements

• Productivity vs. Status– You will get what you measure– What is in the contract?

Offshore Testing

• Executing the Project– Capturing knowledge

• As the offshore test team works through the product, they will acquire knowledge about the product – how do you capture this?

• Turn over rate is currently very high with many offshore companies

– Sharing knowledge• As turnover happens, how do you get new

people up to speed?

Offshore Testing

Noon GMT8:00 EST5:00 PST 17:00 IST 19:00 CST 20:00 JST

WaterlooVancouver London UK Bangalore IN Beijing CN Tokyo JP

Offshore Testing

Finalizing the Project

Offshore Testing

• Finalizing the Project– Do you need to close out the project or

will it be an ongoing effort?• If closing: Retrieve equipment; remove

access; ensure intellectual property is accounted for

• If ongoing: are there budget changes need to ensure proper accounting for resources moving forward? Includes new hardware needed, old hardware returned

Offshore Testing

Lessons Learned

Offshore Testing

• Lessons Learned– Even if you have been doing this as a company

for an extended period of time, there are always lessons learned

– Is there a process in place to capture this and share it with the next project

– How did this company fair compare to the last project or if new, the last company

– What improvements can be made based on the lessons you just learned?

Offshore Testing

Final Questions

Offshore Testing

Your PresentersTracey Clark and Michael

Cookson

Tracey Clark

Tracey Clark is currently a QA manager at Research In Motion. She was worked there for over 2 years and manages a team of over 60 resources including both onshore and offshore.  Tracey previously worked at Sun Life for almost twenty years with ten years of experience in quality assurance and testing activities.

She has supported a variety of business units including Individual and Group Retirement Services and has tested many different technologies including desktop applications, web sites, and mainframe systems. Tracey also champions quality, leads process improvement initiatives, and consults on large, complex and challenging projects.

Tracey also participates regularly in local SQA forums such as SWOSQG and KWSQA.

Tracey can be reached at [email protected].

Mike Cookson

Mike Cookson has been working in the software industry for over 20 years, the majorityOf that time in the Quality Assurance and Testing area. He has worked in the financialSector and most recently testing and deploying e-commerce applications for a worldWide audience.

Mike is currently working at IBM in the Tivoli Software Group, working as the Manager of Operations for the Services Hosting Center.

Mike can be reached at:

[email protected] or Linkedin or through the South Western Ontario Software Quality Group ([email protected], http://www.swoswg.org)