NASSCOM Software Testing Report Secured

Research & Intelligence

Software Testing – Shifting from Functional to Business Assurance

2Research & Intelligence: Software Testing - Shifting from Functional to Business Assurance

Copyright ©2011

International Youth Centre, Teen Murti Marg, Chanakyapuri

New Delhi - 110 021, India

Phone: 91-11-23010199, Fax: 91-11-23015452

E-mail: [email protected]

Published by

NASSCOM, New Delhi

Designed by

CREATIVE INC.

Phone: 91-11-41634301

Printed at

P.S. Press Services

Disclaimer

The information contained herein has been obtained from sources believed to be reliable. NASSCOM

disclaims all warranties as to the accuracy, completeness or adequacy of such information. NASSCOM

shall have no liability for errors, omissions or inadequacies in the information contained herein, or for

interpretations thereof.

The material in this publication is copyrighted. No part of this report can be reproduced either on paper

or electronic media without permission in writing from NASSCOM. Request for permission to reproduce

any part of the report may be sent to NASSCOM.


Som Mittal

President

Over the last 5-6 years, India has become one of the leading destinations for outsourcing of Software

Testing services accounting for 32 per cent of the total global outsourcing share. Export revenues and

the number of employees have doubled over the last four years growing at a CAGR of ~20 per cent. Key

growth drivers include faster time-to-market, compelling cost advantage, evolution of technologies,

availability of skilled talent pool, and increased maturity of Indian vendors off ering end-to-end,

innovative services and solutions. Mature usage of these services is ultimately steering towards defi ned

levels of business assurance, taking quality assurance to the next level.

By 2020, India’s Testing segment is expected to be ~USD 13-15 billion, and exploiting this opportunity

will require all stakeholders – industry, academia, government and associations to collaborate and

promote India as the preferred destination for Testing services.

This report highlights and substantiates with case studies – the evolution of the Testing segment in

India, the current and future market size, traces the changes in business and pricing models, growth

strategies, emerging technologies and their impact on Testing services, the challenges faced by this

segment and initiatives taken to mitigate them. The report also highlights the initiatives that should

be undertaken by NASSCOM to promote visibility of this segment to the world.

We hope that you fi nd this report on India’s Software Testing services segment useful, and we welcome

your feedback and comments.

Foreword


Industry Overview 5

Industry Drivers and Evolution 12

Key Industry Trends 17

2020 Outlook 37

Challenges 39

Summary 42

Contents


Industry Overview

Industry Drivers and Evolution

Key Industry Trends

2020 Outlook

Challenges

Summary

Research & Intelligence: Software Testing - Shifting from Functional to Business Assurance6

Testing services pertain to third-party quality assurance services that are provided in discrete and/or

embedded forms by major global and large regional full-service outsourcers/SIs to enterprises, ISVs,

technology product organisations, or OEM customers.

Software Testing services include:

• The full lifecycle of testing services i.e. from requirement analysis; planning; design and development;

to execution and reporting

• Development tests, systems tests, acceptance test and non-functional testing

• Consulting and advisory services on how to best introduce and conduct testing within an

organisation, assess current maturity of the client, advise on Software Testing tools and help manage

testing projects

Testing1 services addressing risk early and ensuring quality assurance


The global Testing market was worth USD 30 billion in FY2010. Of this, more than 65 per cent is generated

in-house, while the remaining is outsourced. India accounts for 32 per cent of the off shored revenues,

or USD 3.2 billion, thus gaining the position of the largest off shore destination.

The USD 3.2 billion revenue fi gures include revenues from both independent and traditional services.

Traditional services include testing revenues which accrue from large application development, and

product development contracts. Traditional Testing services are bundled with the development process

and is performed by non full-time testing professionals.

This report focuses on the independent Testing market which include revenues from ‘only’ testing

contracts, and which account for nearly 60 per cent of the off shored revenue, while traditional testing

services form the remaining portion of the pie.

India is the largest off shore destination accounting for32 per cent of the pie


While testing is an important part in other services, the Indian independent Software Testing market

is expected to generate revenues of USD 2.3 billion in FY2011, growing at 23.5 per cent over FY2010.

Over the last 5-6 years, India has become one of the leading destinations for outsourcing of Software

Testing services accounting for 32 per cent of the total global outsourcing share. Export revenues and

the number of employees have doubled over the last four years growing at a CAGR of ~20 per cent.

Most of the current testing that is outsourced concerns conventional elements of functional testing,

primarily systems testing, regression testing and integration testing.

Gradual move towards non-linearity through increased automation


• Currently though fi nancial services account for more than half of the share in export revenues,

there is an increasing demand from non-fi nancial services segment like Hi-Tech, Telecom, Retail

and Healthcare and this trend is expected to continue in the future

• Adoption and awareness of independent testing services has been growing, most rapidly in the US,

followed by the broader Americas, Western Europe, and Asia Pacifi c regions. There is an increasing

demand from the non-US markets like Continental Europe and APAC especially Australia, the

Middle East and India

• Large and mid-sized players account for over 70 per cent of this market, but there is substantial

play from smaller, pure-play vendors. Pure-play testing providers (e.g., AppLabs, Maveric Systems,

QA Infotech, Thinksoft) and mid-tier players (e.g., MindTree, Cybage, Patni, Sonata Software),

represent a growing competitive force to the integrated large players in Testing services

• Some of pure-play testing providers, focus wholly on the full continuum of Testing services, while

other outsourcers are focused across a select set of verticals or target product engineering and

development services. Smaller vendors in the industry have made signifi cant gains in off ering

testing as a service to emerging verticals and regions by developing niche, customised solutions

BFSI vertical and the US market are driving growth with pure-play accounting for a third of the pie


Testing services, extend across the software development lifecycle from basic functional level testing

to more specialised testing services. Services can begin from front-end QA consulting services, through

test design, test case development, execution, and project tracking to performance against target

productivity or defect density SLAs. Customers engage in QA consulting services to drive establishment

of test strategy, or the development of centralised, standardised testing processes and frameworks,

and/or best practices to drive and track test planning and execution fl ows.

Current trend shows increased demand on specialised areas in testing such as test automation,

non-functional testing and test consulting services. Key market trends include:

Functional testing achieves average margins while performance testing have higher margins.

• Functional testing account for 75 per cent of the testing lifecycle and is the main growth driver.

Of this, 65 per cent is manual while 10 per cent is automated indicating the move towards

high-end services

• Basic functional testing services are already commoditised, but emerging categories of service

targeted at higher value realisation are evolving rapidly

Shift from traditional functional testing to specialised services


• Specialised testing services account for the remaining 15-25 per cent of the pie with ERP, SOA

testing accounting for 5-10 per cent. This spend is likely to increase as clients expand their focus

from custom application testing to ERP testing, cloud services, etc.

• Consulting focuses on quality assurance, programme management and packaged applications

setting up TCoE for clients



Industry Overview


Key Industry Trends

2020 Outlook

Challenges

Summary


Testing has become inherent across and throughout the lifecycle of production or service delivery. It

has become a change agent addressing risk early on in the lifecycle and continually assuring reliability,

relevance and compliance apart from providing functional acceptance and assurance for the product or

service. Testing has become a value creator and quality diff erentiator for the end product to provide

the required edge to compete in the marketplace of excellence. Leading vendors now position testing

as a standalone service to highlight their focus and service maturity which presents a compelling value

proposition with cost control and value creation.

Organisations around the world using independent testing services providers are benefi ting with

signifi cant improvement in their business strategies that include risk mitigation, validation of new

products/services, being able to support faster time-to-market with reduced test cycles and improving

real-time business performance and monitoring. On the operational front, customers are better able

to meet security audits, develop custom testing solutions that focus on their industry, better use

packaged application testing for ERP implementations and upgrades, leverage transaction testing in

support of revenue assurance and ensure applications' country/region-specifi c readiness.

Improved time-to-market and availability of talent are the key demand drivers


The testing landscape has evolved swiftly both in terms of market demand and vendor capabilities over

the last two decades. While the mid-90’s saw Software Testing as an integral part of the development

lifecycle, by mid-2000, testing as an independent service started emerging. Banks and Telecom vendors

started perceiving testing as an entire business requirement, organisations began accepting testing

as an important part of QA and started budgeting separately for testing. Separate RFPs for testing

were fl oated instead of bundling with development contracts, leading to Indian service providers

spinning off quality assurance into separate divisions. As customers started looking at off shoring of

testing services to include specialist testing services rather than staff augmentation, providers started

moving up the value chain providing high-end services like ERP testing, domain-specifi c testing, etc.

Testing Centres of Excellence (TCoE) were established to provide proof of concept services and other

customised services. With the emergence of cloud-based services, testing vendors also joined the

bandwagon to provide diff erentiated services. Over the years, testing has evolved from being a

functional requirement to a strategic need for the business.

Testing off erings have rapidly evolved, and so have the vendor capabilities


The focus for testing activities is not only on the benefi ts of quality but also on how vendors can

reduce application development cost and speed time-to-market. In today’s application development

organisations, testers are gaining stature as critical elements in the process and delivery team; they

off er a level of focused skills and business awareness that brings a contextual richness to the delivery

process. To maintain this momentum, quality assurance testers must expand their approach to quality

beyond just testing, because functional or performance testing alone is not enough.

Testing organisations are adopting a more holistic approach to testing — ranging from strategic quality

management planning that estimates risks and eff ort for the testing process, to becoming involved

earlier in the process. As testing organisations work more collaboratively with developers, they have

learned technical skills that they can apply in system and integration testing. Developing new skills is

crucial, but tools are also an important part of the equation. Today’s tools that can be applied across

the software development lifecycle provide greater opportunities for quality assurance organisations

to build a complete feedback loop between them and the development team.

Need for testing across the value chain leading to transformative testing services


Independent testing practices has unique characteristics which grew from virtually nothing earlier in

this decade to become one of the fastest growing segments of the IT services segment today. One

reason for this growth is the increasing recognition among customers of how strategic testing can be

for customer-facing, internet-based applications where system failure can mean embarrassment or

worse. Testing services have also grown due to providers separating out testing activity that they once

provided in the context of bundled, end-to-end project services into a distinct service off ering. The

maturity of tool sets from suppliers like Borland, Compuware, HP, IBM, and others have also contributed

by providing potential for greater automation and collaboration between suppliers and customers.

Testing services still lag behind other categories of Application Development and Maintenance (ADM)

services in terms of size and overall customer penetration, but customer buying intention data

foreshadows continuing growth.

Because testing has been frequently undervalued in enterprise IT organisations, internal capability for

testing can be hard to fi nd. With the growth in the external testing services market, trained professionals

with relevant certifi cations are readily available. Access to such talent is a primary motivation for many

customers. Investment in labs or Test Centres of Excellence is a characteristic unique to the testing

segment which also works as proof of concept centres where all the competencies/tools/solutions are

assembled and customised solution is provided.

Testing – an industry with unique characteristics



Industry Overview


Key Industry Trends

2020 Outlook

Challenges

Summary


Diversity of the market multiplied by huge customer expectation and demand have led to the emergence

of delivery models ranging from the traditional staff augmentation to managed Centre of Excellence

(CoE). The delivery models include ability to deliver shared or dedicated outsourced Test Centre of

Excellence (TCoE) models — project-based testing services for staff augmentation off ered in support

of basic or specialised testing services; services being off ered in conjunction with testing-as-a-service

models; process for continuous improvement, best practices, and consultancy services. Providers

are off ering a mix of test delivery models, with many of them also launching the new or enhanced

cloud-based testing services.

Over the years, customers are increasingly focusing on establishing both preventive and proactive

measures in the testing discipline and moving towards a central test CoE model to drive innovative

process improvement into the organisational fabric. The providers work with customers to select the

most eff ective models suitable for their business.

Customer demands paving the way for diff erent business models


With changing delivery models, there also has been an immense change in SLAs and metrics over

the past few years. Initially, most metrics focused on cost reductions and savings, that was achieved

through test automation, labour arbitrage and reducing rework through increasing defect discovery thus

improving the testing economics. The next phase saw metrics focusing on service quality, in addition

to the cost metrics. This involved identifying defects early in the UAT process and providing better

quality services across the value chain thereby reducing the cost. The increasing maturity of Indian

service providers allowed them to provide consistently high quality service delivery, and improving the

user defi ned standards on a regular basis.

However, in the past few years, the industry has seen increased shift to business outcome-based

metrics, where the service providers are expected to assure a certain impact/consequence. In an

outcome-based pricing model, vendors have started to off er outcome-based pricing models to clients

which guarantee pricing levels when pre-decided specifi c criteria regarding service performance are

met. These types of models are usually off ered to clients with whom vendors have had long standing

relationships and where the service levels had reached a stage where the outcome criteria can clearly

be judged. Metrics based on outcome can include the revenue impact on clients, the impact on end

customers of clients, lower test lifecycle time and faster time-to-market.

Leading to signifi cant changes in SLA metrics


High levels of similarity across comprehensive testing service vendors demand that providers clearly

articulate how their advanced frameworks can be a diff erentiator to the customer, how it can improve

assurance (business and technical) at lower cost in shorter time frames with higher predictability.

To attain a leadership position among the abundant service providers, vendors have identifi ed and

started adopting diff erent growth strategies. Leveraging new delivery models, creating their own

IP model to scale up, increase their IP investment in automation across the test value chain, setting

onshore and nearshore centres to tap into the local talent available, creating test labs and off ering them

on hire to customers, forming partnerships with tools/solutions providers are some of the strategies

followed by vendors.

By providing risk mitigating services and maintaining the quality of services, providers are increasingly

driving validation of software to the front of the lifecycle and thus, closely aligning testing services

with business value.

Vendors adopting unique growth strategies to establish leadership


Wipro Testing Centre of Excellence delivering substantial benefi ts


QA Infotech providing globalisation & localisation testing services


TCS-Adoption of latest trends and technology through strategic alliances


Long-term investment in customer relationship, diff erentiated value proposition, cultural affi nity,

awareness of shifting customer pain points, and high resource continuity, all infl uence a provider

selection process for outsourcing services. With increasing confi dence of the customer in the vendor,

the services outsourced also matures over a period of time.

Customer relationship forms an integral part of any engagement. Value of customer partnership

depends on the governance and high-level management and involvement the customer provides

from the client side. Vendors that stay close to their customers, whether engaged in small entry-level

engagements or larger deals, can more quickly identify client dissatisfaction and work to remedy the

problems. Without this vigilance, customer dissatisfaction can build and put a relationship in jeopardy.

Better client management also leads to high repeat business and customer goodwill.

Maturing client relationships key to successful business engagements


The Software Testing industry is undergoing a series of change in service off erings, business and

engagement models and innovation. Portfolios are being expanded to include domain-specifi c niche

service off erings. There is a noticeable shift from traditional functional testing to specialised testing

such as SAP/Oracle, SOA, ERP and performance monitoring testing.

Engagement models have also seen signifi cant movement, with both deal durations and sizes

increase as clients realise the growing importance of outsourced testing services, and Indian vendors

get increased acceptance in the marketplace on the strength of their solutions and transformative

capabilities. Flexible and outcome-based pricing models are gaining increased acceptance, as vendors

develop an unique value proposition around testing as an independent service.

Further, this segment has seen considerable innovation as vendors develop IP (tools, accelerators,

technologies, processes), build comprehensive automated test labs for hire that not only aim to increase

speed-to-market, but also give clients plug and play capabilities around testing as a service.

Key trends highlight Software Testing as a rapidly evolving industry


Maverick — Domain-specifi c testing services (Banking)


Thinksoft — Domain-specifi c testing services (Stock Exchange)


Cognizant’s experience in enabling agility, automated QA and engineering quality discipline


Infi nite Computers using agile practices in a testing project


Cybages’ agile testing methodology for customers


Applabs testing as an independent service


Software Testing is a labour-intensive process with the availability and quality of employable talent

forming the mainstay. The industry needs to consistently invest in training employees as well as

growing the talent pool systematically to a scale that will support the industry’s growth aspirations

over the next few years.

In order to keep this foundation strong, organisations have formulated diff erent strategies and

initiatives to cater to the ever increasing demand for skilled manpower. Offshore vendors are

continuously investing in training, skills enhancement, providing domain-specifi c certifi cations and

hiring qualifi ed personnel. Most Indian vendors have established dedicated in-house training academies

and Centres of Excellence (CoEs). These centres aim to bridge the gap between the present and desired

capabilities of the workforce and can provide training in either specifi c technologies that the organisation

lacks or in specifi c client needs. Organisations have also formed tie-ups with universities and training

institutes to impart specialised training.

To attract skilled talent as testers and to promote testing as a viable career option, many organisations

have chalked out specifi c career path for employees in the organisation and are providing salaries at

par with that of developers.

Skilled workforce forms the backbone of the business


TCS establishing dedicated career paths for testing professionals


Infosys Testing Academy imparting industry-specifi c training


The domestic market revenues from Software Testing during FY2010 was less than USD 200 million.

Though this market is currently substantially small it is expected to cross USD 1.5 billion by FY2020.

With an expected increase in demand for India-specifi c application development, the need for specialised

testing services is bound to increase as organisations look at tactics to enhance their go-to-market

strategies. Availability of employable talent pool with domain-specifi c skill-set will also further drive

growth. A substantial opportunity from the government and SMB sector will provide an impetus to

the growth of the domestic testing segment. With increase in MNCs setting up development centres

in India, for e.g. Automobile majors setting up their design centres in India, the domestic testing

segment is bound to grow further.

The domestic market for testing is currently small, but is poised to grow substantially going forward


ITC Infotech’s Performance Testing for Sales Management Information System (SMIS) for a Indian FMCG company



Industry Overview


Key Industry Trends

2020 Outlook

Challenges

Summary


The Software Testing business presents a signifi cant opportunity to the Indian IT-BPO industry.

The total global Software Testing market is estimated to reach USD 50 billion by 2020. Of this,

USD 20-25 billion is the addressable market. India is expected to capture a leader’s share of about

55 per cent (~USD 24-28 billion) with testing-related services expected to account for 17 per cent of

incremental IT services growth.

Testing services exports can quadruple by 2020



Industry Overview


Key Industry Trends

2020 Outlook

Challenges

Summary


In order to attain the growth predicted, India and the industry will need to take various measures to

mitigate the challenges faced by Software Testing players:

Commoditisation/Comprehensiveness: As a result of the low-entry barriers to provide testing services,

the segment faces an imminent danger of being commoditised unless, players create diff erentiators

in terms of new delivery models, use of emerging and innovative technologies to provide services and

develop domain expertise. Fulfi ling customer expectations and standing out as a specialist amidst the

plethora of service providers is an arduous task for players.

Talent: Increased competition among organisations for a limited employable talent pool has led to

higher attrition rates. Organisations are implementing various employee engagement measures to

attract and retain talent. They have either set-up internal academies and Centres of Excellence or have

tied-up with external training institutions to upskill/cross-skill employees and are making eff orts to

promote testing as an attractive career opportunity. Organisations also need to engage with domain

specialists to cater to the increasing customer requirement.

Innovation: To remain competitive, organisations have to continuously innovate and provide

value-added specialised services. However, the increasing cost for innovating and lack of capabilities

required is a major hindarance.

But to achieve that growth, the industry has to strengthen current measures to address challenges


Branding & Outreach: With the emergence of other low-cost locations providing similar services, there

is a need to create awareness and visibility for promoting India as preferred destination for Software

Testing services. All the key stakeholders including the government, organisations and trade bodies

have to actively initiate steps to take this forward.



Industry Overview


Key Industry Trends

2020 Outlook

Challenges

Summary


To Summarise:


International Youth Centre

Teen Murti Marg, Chanakyapuri

New Delhi 110 021, India

T 91 11 2301 0199 F 91 11 2301 5452

[email protected]

www.nasscom.in

NASSCOM Software Testing Report Secured

Documents