TEST Magazine

INNOVATION FOR SOFTWARE QUALITY

VOLUME 5: ISSUE 6 DECEMBER 2013

www.testmagazine.co.uk

INSIDE: HIGHLIGHTS: TESTA 2013 NEWS:

“THE WORST IS YET TO COME”

TEST AUTOMATION: WHAT’S IN A GLITCH?

TEST

THE EUROPEAN SOFTWARE TESTER

A PRESTIGIOUS EVENT THAT ACKNOWLEDGED AND CELEBRATED SUCCESS...

DECEMBER 2013 | www.testmagazine.co.uk PAGE 3

CONTENTS

NEWS 7 One in four outsourced projects fail

WORLD NEWS 9 China uses apps to provide info

to the public

VIEWPOINTS 12 The strategic advantage of TCoE Vijay Balasubramaniam and Hardeep Garewal

take a look at the developing role of Testing Centres of Excellence (TCoE)…

TESTA ROUND-UP 17 Photos,commentandopinionfromthefirst

ever European Software Testing Awards. This dedicated section of the magazine showcases thefinalistsandwinnersoftheevening,andtheyexplain what this recognition means to them…

TEST DATA 32 Linking test data management

and quality software – Part 2 Srikanth Dora Karam and Narayana Maruvada

explain the challenges with test data preparation and offer a solution…

SECURITY 35 Do businesses understand the

importance of penetration testing? Today’s constantly evolving security threats mean

mere point-in-time testing could unknowingly leave a network or application exposed to an attacker and possibly face a costly, damaging attack for businesses, says John Yeo…

INSIDE THIS ISSUE

TEST AUTOMATION 41 What’s in a glitch? Automation is the key for businesses preparing and

testing against worst-case scenarios. Neil Kinson, discusses the importance of anticipation when it comes to technology testing…

LAST WORD 46 Get off my grass Dave Whalen explains how he’s grown

into his childhood neighbour…

14.INTELLIGENT

TESTING AND WHY IT’S A GAME

CHANGER

17.TESTA

ROUND-UP

Flexible, cost effective resources to meet the peaks and troughs of your projects

Novus

Capita Novus trains new test analysts

in the latest technologies and supplies

them to organisations looking for:

Cost Effective Resourcev Low risk fixed cost

v Cheaper than contractor market

Flexibilityv Benefits of permanent employees

with the flexibility of contractors

Reduced Riskv UK based staff

v Employment responsibilities lie with Capita

v Option to employ at the end of a fixed term

Call: +44 (0) 8456077466Email: [email protected]

www.capita-at.co.uk/novus

Want to find out more?


Flexible, cost effective resources to meet the peaks and troughs of your projects

Novus

Capita Novus trains new test analysts

in the latest technologies and supplies

them to organisations looking for:

Cost Effective Resourcev Low risk fixed cost

v Cheaper than contractor market

Flexibilityv Benefits of permanent employees

with the flexibility of contractors

Reduced Riskv UK based staff

v Employment responsibilities lie with Capita

v Option to employ at the end of a fixed term

Call: +44 (0) 8456077466Email: [email protected]

www.capita-at.co.uk/novus

Want to find out more?

LEADER

© 2013 31 Media Limited. All rights reserved.

TEST Magazine is edited, designed, and published by 31 Media Limited. No part of TEST Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher. A reprint service is available.

Opinions expressed in this journal do not necessarily reflectthoseoftheeditororTESTMagazineoritspublisher, 31 Media Limited.

ISSN 2040-01-60

EDITORSophie-Marie Odum [email protected] Tel: +44 (0)203 056 4599

TO ADVERTISE CONTACT:Sarah Walsh [email protected] Tel: +44(0)203 668 6945

PRODUCTION & DESIGNTina Harris [email protected]

EDITORIAL & ADVERTISING ENQUIRIES31 Media Ltd, 41-42 Daisy Business Park, 19-35 Sylvan Grove, London, SE15 1PDTel: +44 (0) 870 863 6930

Email: [email protected] Web: www.testmagazine.co.uk

PRINTED BYPensord, Tram Road, Pontllanfraith, Blackwood, NP12 2YA

THIRTYONE

WHAT AN END TO THE YEAR?!As we approach the end of another year, many people and businesses reflect on what has worked well, and what can be improved on for 2014.

owever the software testing industry is in high spirits as 2013 has ended spectacularly with the first ever European Software Testing Awards (TESTA), which saw companies of different sizes, around Europe, receive

recognition and awards for their tremendous software testing and QA efforts.

Therefore, it’s only right that we dedicate most of this issue to this fantastic ceremony. It serves as a reminder of what a great evening it was, offers a keepsake for those who attended and shows others what they missed out on! For more coverage of the evening, you can view video highlights at: www.softwaretestingawards.com

Looking back on the 2013, there have been many trends, most notably mobile testing, which has really come to the fore; its adoption has risen from 31% in 2012 to 55% in 2013, according to the World Quality Report 2013-14. I’m sure we wouldn’t be wrong to assume that this will only continue to gather momentum in the New Year. However, as will the pressures to bring products to market faster; satisfy the growing consumer demands; and the necessity to comply with data security regulations and requirements.

Through the delivery of cutting-edge, up-to-date information and dynamic content, via our online and print channels, TEST Magazine will continue to deliver valuable information, advice and tips to help the software testing community foresee trends and meet expectations.

We will also continue to provide recognition and acknowledgement of the software testing industry via our awards programme and through our newly launched National Software Testing Conference, which is taking place in May 2014.

This Conference, sponsored by Borland Software, will offer the community the perfect opportunity to come together, share knowledge and network. We already have some great speakers confirmed, including winners and finalists from TESTA, as well as figureheads from Expedia, Waitrose and Barclaycard, who will all share their expertise. For more information, please visit www.softwaretestingconference.com.

Throughout 2014, we look forward to providing you, our loyal and ever-growing readership, with the latest news and comment from the industry, whilst we continue to help steer the industry forward.

Thank you for all your support during 2013 and all the best for the New Year.

Do you want to write for

TEST magazine? Please email sophie.

[email protected]

H

Sophie-Marie Odum Editor

Don't take testing lightly; simple, small things can cost a great deal more...

Where is the margin for error?

Let us help you assure your business and your peace of mind

In a fast-evolving marketplace and ever-changing customer demands, there is very little margin for error. Lack of comprehensive and holistic assurance can lead to loss of business reputation,

and have a negative impact on the top line.

TCS Assurance Services can help assure your business.

Write to us at [email protected] for a personalised discussion, or visit www.tcs.com/assurance to learn more.

�0

�5

�2�5

�7�5

�9�5

�1�0�0

�0

�5

�2�5

�7�5

�9�5

�1�0�0

�0

�5

�2�5

�7�5

�9�5

�1�0�0

�0

�5

�2�5

�7�5

�9�5

�1�0�0

�T�C�S�_�A�s�s�u�r�a�n�c�e�_�P�o�s�t�e�r�_�A�4�_�2�9�1�1�1�3�_�Q�R�_�P�r�e�p�r�e�s�s

�F�r�i�d�a�y�,� �N�o�v�e�m�b�e�r� �2�9�,� �2�0�1�3� �4�:�5�1�:�1�6� �P�M


NEWS

ONE IN FOUR OUTSOURCED PROJECTS FAIL

31% of outsourced projects have run into service level or time issues and 23% have failed to deliver on the final requirements, threatening 31% of CIO jobs. This is according to an independent global research study undertaken by Vanson Bourne and commissioned by Borland, a Micro Focus company.

Of the 590 CIOs and IT directors polled from nine countries around the globe, more than half (57%) described some of their outsourced testing and development projects as “unmanageable”, “an embarrassment”, “a nightmare” or “a total failure”. The majority of respondents (55%) cited “too many changes to the requirements during the project” as the main reason for project overruns or failure to deliver on the final requirements. This is a serious issue given 47% of organisations change the

specification of work being done by their outsourced vendor at least once a fortnight or more frequently.

When questioned about their outsourcing vendor relationships during projects with time or service level issues, over half of CIOs (53%) said they felt let down by their partner. Surprisingly, 43% of these said although it was far from ideal, issues were par for the course when using outsourcing vendors. Overall, 84% of survey respondents claimed outsourced development and testing projects had created negative issues for their organisation, ranging from delays to customer products (39%), the ability to protect company IP (29%) and reputation (25%), and even an impact on company revenues (12%).

A shocking 98% of CIOs confirmed some form of additional in-house work was needed following the final delivery of

outsourced projects.

Chris Livesey, Borland Vice President at Micro Focus, said, “The research highlights that the relatively poor outsourcing results are often caused by a lack of investment in the processes for requirements management and test specification, and this is something we see consistently in the market place today.

“However, the results could be greatly improved by investing in a much clearer statement of requirements and their associated test cases early in the project, enabling both the end client and the service provider to more accurately estimate the project schedule, risks and costs at the beginning of the contract (…) With this early and continual visibility these partnerships would work much more effectively.”

POLL RESULTS

Last month we asked, "DO YOU THINK OFFSHORE TESTING IS DECLINING?" at www.testingmagazine.com

THE RESULTS WERE:

BUSINESSES MUST HAVE EFFECTIVE DATA MANAGEMENT POLICIES IN PLACE

From supermarket chains to high street banks and online stores to large multinationals – effective data management is key to the success of any business, yet the author of the latest book to be published by BCS, The Chartered Institute for IT, questions whether organisations “fully appreciate the importance of data as a vital corporate asset and are taking appropriate steps to ensure that it is fit-for-purpose.”

Keith Gordon, author of Principles of Data Management: Facilitating information sharing (Second edition), explained, “The key to the provision of high-quality information and the sharing of information between systems is to have an effective policy for corporate data management in place. Yet very few senior business and IT or IS managers have heard of data management, let alone have an effective policy for data management in place.”

The book aims to help organisations ensure they fully understand:• The difference between data and information and their

importance to an enterprise. • Data modelling; both at the project level and the

corporate level. • The activities necessary to manage data as a

corporate resource. • How to achieve good quality data. • The technical environment within which data management

has to operate.

The book is business-focused, providing the knowledge and techniques required to successfully implement the data management function. It examines all areas of data management, including database development, data quality and corporate data modelling. This new edition also covers web technology and its relation to databases, and includes material on the management of master data.

YES NO

For the latest news, visit softwaretestingnews.co.uk

and follow us @testmagazine

60%

40%

To answer this month's poll, please visit: www.testingmagazine.com

TWO IN FIVE SURVEYED FEEL THEY HAVE MORE TO GIVE

More than two in five of Britain’s IT professionals feel they have more to give in their career in order to reach their full potential.

According to research by Randstad Technologies, Britain’s IT staff are showing they are not prepared to rest on their laurels, as 44% of IT staff feel they have not yet reached their full potential. This ambition within the IT sector comes despite the fact the industry has the highest levels of professional fulfilment compared to other sectors in the UK with 73% of IT professionals describing themselves as professionally fulfilled – well above the UK average of 62%.

Mike Beresford, managing director of Randstad Technologies, commented, “Positive signals for the UK economy seem to be appearing every week which is fantastic news if we’re to achieve a sustainable recovery. What is even more encouraging though is that despite being the most professionally fulfilled sector in the UK, those working in the sector have the ambition to advance even further. It is an exciting time for the tech sector in the UK and the dynamism of the industry is reflected in the goals of the staff that are driving it forward.”

More than a third (38%) of staff in the sector feel that it is personal characteristics that are the most important

influence on achieving your potential: 43% view determination as the most important personal characteristic; 39% see adaptability as the next most important character trait; and a strong work ethic was viewed as the third (39%).

27% feel that training, development, good leadership and culture are the keys to success, but, in contrast, 41% of IT professionals see the employer as the biggest barrier to an individual’s success, compared to 25% who see it as the individual.

Beresford, added, “While for the wider population the employer is seen as less of a barrier to success and fulfilment, for the majority of IT professionals it is the employer who supplies the tools of their trade. If these are not up to scratch it may be harder for IT staff to feel they are able to reach their potential.”

DECEMBER 2013 | www.testmagazine.co.ukPAGE 8

NEWS

“THE WORST IS YET TO COME”

Cyber-crime and malware attacks will increase during 2014 as criminals capitalise on recent successes and continue to use technology for illicit gain, according to AppRiver’s threat forecast for 2014.

It warns that many of the security headlines of 2013 are just the precursor of things to come and predicts that, having proved lucrative, many criminals will continue to invest time and effort infecting devices and holding data to ransom; there will be devastating repercussions following Adobe’s ColdFusion and Acrobat source code liberation, mobile malware will continue to spread and everyone’s liberty is at risk from state-sponsored attacks.

Jon French from AppRiver explained, “Thanks to the media hype and the sheer effectiveness of malware such as Citadel and CryptoLocker in 2013, we should expect to see a continued if not an increased use of Ransomware well into 2014.

“The only thing that would possibly hinder the appearances of more Ransomware on the horizon would be the capture of the CryptoLocker

group by authorities. Another cause for concern is Adobe’s recent breach. Obviously there are immediate repercussions for the individuals whose credentials have been stolen, but this is just the tip of this devastating iceberg.

“In a case of stolen software or source code, there is also a large risk of a hacker knowing the ins and outs of the software thereby allowing them to write malicious code aimed at weaknesses they find. With the source code at their power, it’s a fair assumption that we’ll be seeing these exploits come to light in 2014.”

Speaking specifically of the highly publicised increased frequency of state-sponsored attacks and cyber-espionage, French continued, “Every day, critical infrastructure and organisation entities face state-sponsored cyber-attack.

"Far less common is for nation states to admit being behind them, but that doesn’t mean we’re naive enough to believe it’s not happening. As more countries become equipped, empowered and emboldened to orchestrate these attacks we can only

expect the frequency and severity to increase.”

Before his untimely passing earlier this year, Barnaby Jack warned that hackers could potentially take control of medical devices with deadly intent. While AppRiver does not predict an epidemic of pacemaker murders – though it does not deny the possibility – it is concerned that hackers who enjoy “pranking” will target the ever increasing diverse array of devices connected to the Internet, such as Internet-controlled thermostats, causing, at best, inconvenience and potentially financial implications.

There are precautions users can take to avoid becoming the next victim of cybercrime. Anti-virus software, user education and implementing sound security practices can help mitigate online threats. But as French warned, “Security must always be at the forefront of people’s minds (…) Educating individuals to spot phishy emails and recognise file extensions can mean the difference between a clean computer and a malicious one.”

Sogeti - a leading provider of Software Testing services:

Ranked #1 for Outsourced Testing Services, Ovum 2011 Winner of Multiple Microsoft Partner Awards in 2011, 2012 & 2013

Winner of HP Award for Innovation in EMEA 2012Winner of multiple IBM Beacon Awards in each of the last 6 years

Want to fi nd out more?

Call: +44 (0) 20 7014 8900 Email: [email protected]

www.uk.sogeti.com

Winner: Testing Innovator of the Year and Finalists in Young Tester of the Year category


NEWS/WORLD NEWS

PARTICIPANTS AWARDED WITH “CERTIFIED SNAP PRACTITIONER” STATUS

The International Function Point Users Group (IFPUG) is pleased to announce the results of the world’s first certification exam for the Software Non-functional Assessment Process (SNAP) size measurement, held at ISMA 8, in Rio de Janeiro, Brazil.

20 exam participants were awarded the title of “Certified SNAP Practitioner.” This first class of certified practitioners by the IFPUG, joins the hundreds of Certified Function Point Specialists across the globe.

SNAP provides a quantifiable measure for the Non-Functional Requirements (NFR) which allows organisations to build historical data that can be referenced to assist in decision making for the technical and/or quality aspects of applications. Non-functional requirements which, until the advent of SNAP, have not been consistently measureable, according to the group, can contribute

substantially to the effort required to bring software projects to market. The inability to measure non-functional size has been one of the major reasons that many software

projects are late or over budget, according

to the organisation.

It's been said that the SNAP standard provides organisations using IFPUG measures with a competitive advantage to those using other software sizing methods, which do not account for non-functional software size.

The Software Non-functional Assessment Process (SNAP) Assessment Practices Manual is released under a Creative Commons 3.0 Attribution-NonCommercial-ShareAlike License. IFPUG is the first international functional sizing standards organisation to release a sizing standard under the Creative Commons License.

BULGARIA CERTIFIES SOFTWARE TESTING LAB FOR ONLINE GAMBLING

In late October, the Bulgarian State Gambling Commission made headlines for banning numerous online poker sites. However, the Government recently released a list of three certified gaming equipment and software testing laboratories — BMM Testlabs, Gaming Laboratories International (GLI), and Bulgarian Institute Metrology.

Only one online operator, Malta-based Eurofootball, has received a license to offer gambling in the country.

20 EXAM PARTICIPANTS

WERE AWARDED THE TITLE OF

“CERTIFIED SNAP PRACTITIONER”

#1 SCORE IN PERFORMANCE TEST OF ANTIVIRUS SUITES

Avira has earned the top score in AV-Comparatives’ Performance Test of antivirus suites, which measures the impact of Internet security software on computer system performance. In total, 22 competitive products were tested, and, AV-Comparatives awarded Avira its Advanced+ three-star certificate.

The Performance Test measured the speed of common computer operations like opening a PDF or Office document, downloading or copying a file, installing

and uninstalling applications, archiving and unarchiving storage files, and encoding and transcoding multimedia files. The test then compares these speeds with and without the antivirus security software running. In every situation, Avira’s security suites earned the top score.

IN TOTAL, 22 COMPETITIVE

PRODUCTS WERE TESTED, AND, AV-COMPARATIVES

AWARDED AVIRA ITS ADVANCED+ THREE-

STAR CERTIFICATE

CHINA USES APPS TO PROVIDE INFO TO THE PUBLIC

China Software Testing Center (CSTC), operated under the Ministry of Industry and Information Technology, has revealed that a quarter of the country’s government ministries, and 31% of provincial-level governments, have developed software applications for smartphones.

As of 15th November, 13 out of 52 government ministries and departments had developed such mobile applications to provide information for the public. However, local governments on the

municipal and township-level are less “mobile,” with only 11% and 3.8% respectively having

such applications.

The increasing use of mobile applications in government services is expected to better cater to China’s 1.22 billion mobile phone users. But, the Center noted most applications are limited to providing news and basic information, and that some

applications are susceptible to software code tampering, user information leakage and other

security problems.

Easily Integrate With Your Current Tracking Tool

• Create and manage test definitions that you can share across multiple environments and teams.• Define testing targets for an entire release and compare your actual results with your estimates using DevTest summary reports.• Schedule and assign tasks using the DevTest planning wizard.• Execute tasks, search for existing defects and submit new defects into DevTrack without having to leave DevTest.• Link test tasks to defects or change requests and track each test task’s history.• Analyze test results with buil• Analyze test results with built-in presentation quality reports.

DevTest has out-of-the-box integration with most popular defect and development tracking tools, including: JIRA®, Bugzilla®, Mantis® and DevTrack®.


VIEWPOINTS

THE STRATEGIC ADVANTAGE OF TCoEVijay Balasubramaniam and Hardeep Garewal of ITC Infotech take a look at the developing role of Testing Centres of Excellence (TCoE)…

f you look at the genesis and evolution of the idea of the Testing Centre of Excellence (TCoE), they’re trying to take the value proposition for the customer to the next level by addressing cost, quality and schedule.

Prior to TCoEs, vendors provided a capability on demand for a required period of time. The value mainly derived from cost advantages; as it was often much cheaper to use a services partner for these specific testing services. Let’s look at banking for example.

Initially, a bank might wish to implement an IT project in a certain way, so it would go out and look for testing and development partners to deliver what it needed in a cost-effective way. The project management and the requirements analysis, etc, would be the responsibility of the customer while the contractor would provide the technical capabilities for testing and development.

Eventually after many projects have been completed, the client-vendor relationship matured with the vendor developing an understanding of all the aspects necessary to deliver a complete high quality testing service. These aspects include an understanding of the business domain; application knowledge; process capabilities; and programme management capabilities, as well as the technical aspects of testing tool-related capabilities like automation, performance and security. With this knowledge in place, the outsourced tester can build competencies which can be offered as a pre-packaged service to the customer.

SERVICE LEVEL AGREEMENTSToday when we talk about a Testing Centre of Excellence what we mean is an end-to-end solution for the customer. So if the customer is planning the introduction of new software infrastructure or a specific business application, they can be assured that the software will work as it should with no production defects, with faster time-to-market, hitting the schedule and cost service level agreements (SLAs).

The services partner should understand what the client is aiming to do and has the people, processes, tools and technology in place to ensure that the end product is exactly how the client wants it. The entire Centre process

is determined by SLAs. Customers can clearly see how stable the application is post-testing and they can view the project from an SLA perspective:

• Were there any post-production defects detected? • Was there any downtime on account of

software quality? • What was the adherence to the agreed schedule? • What was the adherence to the agreed budget?

TCoE iN AN AGILE WORLDAlthough strictly speaking, the TCoE, being a separate entity, can never be truly agile. It can add value in an agile testing environment by taking care of the parts of the testing process that can only be done once the product has been completed. The TCoE can take care

of all the non-functional aspects of quality testing like performance, security, disaster recovery

and operational acceptance testing. These things can only be done at the end of the

development process and not on an agile daily basis. The TCoE can add value here by offering alternative services such as performance testing as a service.

A good example of where TCoEs can add value is in the retail space where “holiday

readiness” is a crucial factor. During the holiday season, retailers sell the most so they

need to know that their IT infrastructure can scale-up to cope with during the peak in sales.

Testing whether the software can cope with the demand can be done by a TCoE that understands the retail business and knows exactly the parameters that are likely to change in the run up to the holiday season.

NO WORRIESFrom the customer’s perspective, using a TCoE completely takes away their responsibility and accountability for quality. When customers come to a Centre they needn’t worry about how the requirements are going to be handled, or how the change is going to be managed, nor about which techniques are the best to use for any particular job or what automatic test scripts need to be created and whether they need to be reused in future.

Centres also bring the advantages of an existing library of useful software and tools to bear, as well as a potentially broader range of testing skills to use on the project.

I

TODAY WHEN WE TALK ABOUT A

TESTING CENTRE OF EXCELLENCE WHAT WE MEAN IS AN END-TO-END SOLUTION FOR

THE CUSTOMER

VIJAY BALASUBRAMANIAM GLOBAL HEAD OF TESTING PRACTICEITC INFOTECH

HARDEEP GAREWAL PRESIDENT, EMEA OPERATIONSITC INFOTECH

Burn Your Spreadsheets

Manage product testing with TestTrack TCM

• Achieve real-time visibility into testing progress.• Reduce risk and achieve greater confidence in the release.• Gain more time to plan projects and manage your team.

TestTrack TCM tracks all details of product testing for you, including test cases, suites, runs, and results. Now you can have real-time visibility over your testing effort and more time to plan and manage.

© 2013 Seapine Software, Inc. All rights reserved.

Download a free, 30-day trial of TestTrack TCM at www.seapine.com/testuk


INTERVIEW

Sophie: What is the future outlook for Assurance Services? What trends do you see enterprises adopting in this space?

Siva: There are many exciting developments that are taking place in the assurance and testing space. In the context of test automation, the future lies in intelligent testing. A smart, intelligent quality assurance and production handshake, encompassing holistic operations and leveraging engineering – quite sophisticated and different from what is currently available. Creating a virtuous cycle between production, test and dev is the future – a huge, paradigm shift from the way traditional testing is currently done.

SO: Can you tell us more about this paradigm shift? What is intelligent about intelligent testing?

SG: Conventional ways of testing look largely at automation of test “execution”. Historically, a lot of effort and tooling has happened around automation of test cases. When we talk about intelligent testing, it is about automation being extended across the full lifecycle: across demand management; test case generation (straight from requirements); automated provisioning of test environment; test data creation and management; and then, automation of test execution.

An intelligent testing system is all about bringing an intelligence arising out of automation across the full lifecycle as opposed to it being siloed in just execution.

Now that’s a big change from what’s been happening in the past and how we see the future to be.

SO: How is intelligent testing different from regular test automation?

SG: Intelligence comes about when you look at automation across the whole value chain. When we talk of intelligence coming about in the way we do testing, it is about the ability, in a very automated and engineered way, to

INTELLIGENT TESTING AND WHY IT’S A GAME CHANGERSophie-Marie Odum has a candid discussion with Siva Ganesan, Vice President and Global Head, Assurance Services, Tata Consultancy Services (TCS), on the concept of intelligent testing and why it’s revolutionising the testing and assurance world.

SIVA GANESAN VICE PRESIDENT AND GLOBAL HEAD,

ASSURANCE SERVICES, TATA CONSULTANCY SERVICES (TCS)


INTERVIEW

have the environments provisioned correctly; to have the test data created; to have the test components and the harnesses to be instrumented in such a manner that there is a lot of agility baked into the testing processes.

The intelligence lies in the superior engineering which looks at automation from an assembly line perspective.

SO: How can intelligent testing deliver value to the business?

SG: The value comes from the assembly line approach to testing. If you look at parallels from other industries such as manufacturing, the automation process is all about how engineering and fabrication happen in an assembly line in an automated way. And if you draw a parallel to the world of testing, what we are trying to do is to ensure end-to-end automation from a test assembly perspective across requirements, the whole act of test case generation, test data generation and environment provisioning. The idea being: the more you automate, the less you have chances of error and therefore, more certainty, more value in terms of improved time to market, lower cost of quality and greater production stability.

SO: What have been the drivers for intelligent testing?

SG: Over the last few decades, we have studied the market and have found common concerns that enterprises have shared with us. “How can we have the environment set up done without a wait time”; “Can we look at ways to reduce the tremendous amount of time taken by my testers to create test data”; “How can we automate our testing cycle?”; “How can I minimise risk by protecting my data from my service?”; “We need to ensure cost reduction with almost no rework”.

Intelligent testing was conceptualised to address these concerns in a holistic, end-to-end manner.

SO: Why is intelligent testing going to become indispensable for organisations in the future?

SG: The whole market we live in today is about better, faster and cheaper. And everyone wants to get there first. Everyone wants to launch everything real-time at least once a week, if not five times a week. In this era where we live in such fast refresh cycles, it is inevitable that one

THE WHOLE MARKET WE LIVE

TODAY IS ABOUT BETTER, FASTER AND CHEAPER.

AND EVERYONE WANTS TO GET THERE FIRST. EVERYONE

WANTS TO LAUNCH EVERYTHING REAL-TIME AT

LEAST ONCE A WEEK, IF NOT FIVE TIMES A

WEEK


has to regress and automate in a hurry. And for that, end-to-end automation, which is holistic and intelligent, is inevitable.

SO: What are some of the key considerations for enterprises in the context of test automation?

SG: Enterprises need to be able to question whether they are exploring the frontiers of assuring value to customers; whether they are assuring that all applications are error free; that everything is being done right the first time; that despite increasing IT complexity, they can always deliver faster and better to end customers at the lowest optimum cost.

They need to see whether their test automation systems are helping them assure error-free business applications and transactions; whether they are able to deliver change into production every time, five times a week, as opposed to waiting quarters on end to launch the basic functionality into production. And, most importantly, if they are viewing testing as a lever to positively impact the balance sheet.

And, of course, despite the complications in today’s architecture comprising social, media, cloud, analytics and more, are enterprises able to ensure that the best is always delivered to their end customers?

SO: You have often spoken about the parallel between Formula 1 racing and assurance. Can you tell us a little bit more about that?

SG: When watching a race, it is amazing how, in milliseconds, pit stoppers come out, change spare tires, check whether they work and say it is a “go” and off goes the car and the driver, perhaps to even win the race!

Everybody wants things faster, cheaper and better, and

just as in the case of Formula 1, everyone wants to reach the podium first. The sheer velocity at which these cars hurl themselves down the race track is perhaps akin to the speed at which most businesses need to hit the market to attain (or maintain) competitive advantage. And just as in Formula 1, a single defect at that speed could be disastrous for the system, the enterprise and indeed, the brand.

The high velocity, the need for agility, the need for supreme quality and the need to be as defect-free as possible – testing, assurance and Formula 1 are all dependent on world class, sophisticated engineering and instrumentation.

SO: A last, curiosity question. It’s interesting that TCS refers to its QA and testing arm as “Assurance Services”. Would you like to tell us why you use the term “assurance”?

SG: I’m sure you’ve heard the famous parable about the person who believed he was laying a brick wall, and the one who believed he was building a cathedral. For us, the word “assurance” denotes a mindset. Testing today is not just about “testing”. It’s about “assuring” the business through comprehensive, smart and, may I add, intelligent testing. Quality assurance and testing represent value waiting to be unlocked and not just defects waiting to be discovered!

THE SHEER VELOCITY AT WHICH

THESE CARS HURL THEMSELVES DOWN THE RACE TRACK IS PERHAPS

AKIN TO THE SPEED AT WHICH MOST BUSINESSES NEED TO HIT THE MARKET TO ATTAIN (OR MAINTAIN)

COMPETITIVE ADVANTAGE

Around 60% of respondents see significant automation challenges in test environment management and test data preparation in their current software testing lifecycle

Source: Survey conducted in October 2013 during a EuroStar-TCS webinar on “Test Smart. Test Intelligent. GenNext Automation”.

More than 55% assurance and testing practitioners think that an intelligent testing platform, which integrates their existing tools, can help them increase test effectiveness

INTERVIEW

57%

30%

9%

4%

Signi-icantly

Moderately

Marginally

Not at all

28%

32%

53%

60%

61%

Service Virtualization

Test Execution

Test Design

Test Environment Management

Test Data Preparation

CELEBRATING TECHNICAL EXCELLENCE

THE EUROPEAN SOFTWARE TESTING AWARDS



Headline Sponsor

20

13

Round -up

I was very honoured to be asked to host the inaugural European Software Testing Awards. In common with software testers, as a voice-over artist, I spend the majority of my time behind the scenes, away from the glare of the spotlight. So being out in front was a new experience for me. I needn't have worried. The attendees were more than welcoming and it was a real delight to spend time with such a

motivated, professional bunch. I thought the evening was a great success.

Peter Dickson, the man behind the most famous voice in Britain and host of TESTA 2013

Develop the skills and knowledge required to work effectively in an agile project environment.

Verify your ability to deliver more efficient testing projects with Certified Agile Tester®, the latest addition to our internationally recognised professional certification portfolio.

bcs.org/agile_1

Take the agile approach

BC2

33/LD/A

D/051

3

© BCS, The Chartered Institute for IT, is the business name of The British Computer Society (Registered charity no. 292786) 2013The Certified Agile Tester® scheme is a trademark of iSQI.

bc233_ld_ad_itnow_ma 17/05/2013 15:46 Page 1


TESTA ROUND-UP

Amazing, elegant and special… Just a few choice words to describe the first ever European Software Testing Awards ceremony, which took place at the luxurious London Marriott Hotel, Grosvenor Square. After an incredible and exciting build up, Wednesday 20th November 2013 has certainly gone down in history as a day to remember for the software testing industry.

The inaugural, sold out ceremony saw hundreds of software testing professionals don their evening gowns and black-tie suits, as they came together to honour and celebrate companies and individuals who have accomplished outstanding achievements in the software testing and quality assurance market.

There was a contagious buzz of laughter and conversation throughout the evening as guests enjoyed catching up and celebrating with peers and colleagues over dinner and fine wine. Regardless of being a winner or finalist, guests were in high spirits that there was finally an awards programme that recognises an industry which rarely gets the credit it deserves. Twitter was awash with pictures and updates #TESTA throughout, which further amplified the event, an event which the industry is still talking about, and will do for a long time to come.

The evening was hosted by famous voice-over artist, Peter Dickson, who kept guests entertained. It was great to see the man behind the most recognisable voice on British television host an awards ceremony for his IT counterparts. Similar to software testers, voice-over artists work behind the scenes to ensure everything runs smoothly, front of house; without Peter would the X-Factor be the same, or Britain’s Got Talent? The same goes for software testers in their respective industries as they are the gatekeepers to software quality. TESTA provided the perfect opportunity for professionals to celebrate and promote excellence, best practice and innovation, in what we believe is one of the most critical areas of IT.

The aim of such an event was not only to reward and recognise though, but to inspire and motivate professionals, which are the essential ingredients for success, and necessary to attract new and bright talent, who will steer the future direction of the industry.

The evening was a resounding success and we, and many others, are already looking forward to next year! The next series of pages are dedicated to the winners and finalists of the evening. For your quick reference, please find below the list of TESTA 2013 winners.

FOREWORD: CHAIR OF





Headline Sponsor

JUDGING PANEL

The TechExcel Best Agile ProjectSopra Group & Student Loans Company

The eggPlant Best Mobile ProjectLloyds Banking Group in partnership with Cognizant Technology Solutions

The Sogeti Best Automation ProjectBrickendon Consulting

Best Overall Testing Project – Public SectorKnowit Oy

Best Overall Testing Project – Finance SectorLMAX Exchange

The Capita Best Overall Testing Project – Retail SectorJohn Lewis IT

Best Use of ToolsTechMahindra

The Thinksoft Testing Team of the YearLloyds Banking Group in partnership with Cognizant Technology Solutions

Testing Management Team of the YearClose Premium Finance

Green Testing Team of the YearCognizant Technology Solutions

Young Tester of the YearAndrew Thomson, Cognizant Technology Solutions

The UKTB Testing Manager of the YearChris Comey, Testing Solutions Group

Testing Innovator of the YearBarry Weston, Sogeti

Test Champion of the YearCraig Ian Thomas, Cognizant Technology Solutions

The BCS Best Overall ProjectTechMahindra

Best Overall Use of TechnologyJaspersoft

Most Innovative ProjectValidSoft UK

The Borland European Software Testing AwardJohn Lewis IT

Best NewcomerCode Factory Group

Best Crowd ProjectTestbirds GmbH

Lifetime Achievement AwardPaul Gerrard

Sophie-Marie Odum, Chair of the TESTA judging panel

Develop the skills and knowledge required to work effectively in an agile project environment.

Verify your ability to deliver more efficient testing projects with Certified Agile Tester®, the latest addition to our internationally recognised professional certification portfolio.

bcs.org/agile_1

Take the agile approach

BC2

33/LD/A

D/051

3

© BCS, The Chartered Institute for IT, is the business name of The British Computer Society (Registered charity no. 292786) 2013The Certified Agile Tester® scheme is a trademark of iSQI.

bc233_ld_ad_itnow_ma 17/05/2013 15:46 Page 1


FOREWORD:





Headline Sponsor

The continually changing technology landscape presents new challenges to solve, and often requires new thinking and imagination, and the rate of change on software projects is faster than ever. At the same time, there is a business requirement to minimise risk and cost, delivering the maximum control and resilience in both the process and the product.

At the 2013 European Software Testing Awards (TESTA), it was clear that software test teams, from a number of industries and countries, are managing to achieve this balance, and the category winners, in particular, demonstrated an unprecedented level of skill, dedication and creativity.

I have the privilege to speak to as many as 50 companies around the world each month about their business, and to work together with them building solutions that deliver the competitive advantage, customer satisfaction and operational efficiencies that they need. In that work I see a number of trends in the software development market, and the judging panel and I saw the same trends within the competition entries.

The first is the establishment of disciplines around testing in agile projects, and it’s evident that teams are still very much “finding their way” here as they strive to strike a balance between speed and control. The second is the emergence of “supply chains of software development”. We may see a major shift in the software industry as it follows a similar path to other manufacturing industries in the evolution of its supply chains. Outsourcing of software development and test will continue to increase – our research shows over 20% of all projects are already outsourced, with a further 20% planned over the next two years – and this supply chain thinking will drive significant change in the way that work is designed,

managed and delivered.

Change in recent years to increasingly distribute, accelerate and de-formalise does not lend itself well to managing these emerging supply chains, and so we will see this being adapted to allow the necessary levels of change management, transparency and collaboration. The companies who will be truly successful in their software projects are the ones who successfully achieve this balance, developing highly effective software supply chains with process and tooling that are designed for the purpose.

The third “trend area” was that of the rate of change of technology – platforms, platform versions, delivery models, cloud and mobile – which surfaced some interesting scenarios for the tester given the resulting fluid nature of today’s projects. The rate of change within software development projects is at an unprecedented level, driven mainly by this rapidly evolving technology landscape.

Despite these challenging trends, it is tremendously encouraging to see the levels of success today’s teams can still achieve, and the case studies the judging panel were presented are testimony to that. The winners have definitely set a great example to the testing industry in Europe for best practice, innovative solutions and dedication to a successful outcome of a project.

Congratulations again to all of the category winners. I am already looking forward to celebrating with next year’s winners at the 2014 awards ceremony!

Chris Livesey, Worldwide VP of Sales for Borland

TESTA ROUND-UP Headline Sponsor

ToseeashortinterviewwithChris,filmedattheTESTAceremony, please visit www.softwaretestingawards.com

HEADLINE SPONSOR


TESTA ROUND-UP

Sopra Group & Student Loans Company

The TechExcel Best Agile Project

Pictured with Peter Dickson (left) and Tieren Zhou (right)

Lloyds Banking Group in partnership with Cognizant Technology Solutions The eggPlant Best Mobile Project

Pictured with Peter Dickson and Antony Edwards (left)

Brickendon Consulting

The Sogeti Best Automation Project

Pictured with Peter Dickson (left) and Darren Coupland (right)

TechMahindraBest Use of Tools

Pictured with Peter Dickson (left) and Peter Hyams (right)

Close Premium FinanceTesting Management Team of the YearPictured with Peter Dickson (left) and Karen Thomas (right)

Cognizant Technology SolutionsGreen Testing Team of the Year

Pictured with Peter Dickson (left) and Lisa Donovan (right)

Knowit Oy

Best Overall Testing Project in the Public Sector

Pictured with Peter Dickson (left) and Grant Farrell (right)

Craig Ian Thomas, Cognizant Technology Solutions Test Champion of the Year



TESTA ROUND-UP

LMAX Exchange

Best Overall Testing Project in the Finance Sector


John Lewis ITThe Capita Best Overall Testing Project in the Retail SectorPictured with Peter Dickson (left) and Mark Roberts (right)

TechMahindraThe BCS Best Overall Project

Pictured with Peter Dickson (left) and Debbie Archer (right)

Lloyds Banking Group in partnership with Cognizant Technology Solutions

The Thinksoft Testing Team of the YearPictured with Peter Dickson (left) and Anand Vyas (right)

Chris Comey, Testing Solutions Group

The UKTB Testing Manager of the Year

Pictured with Peter Dickson (left) and Geoff Thompson (right)

JaspersoftBest Overall Use of Technology


Andrew Thompson, Cognizant Technology Solutions

Young Tester of the Year


ValidSoft UK

The Most Innovative Project



TESTA ROUND-UP

Code Factory GroupBest Newcomer


John Lewis IT

The Borland European Software Testing Award

Pictured with Peter Dickson (left) and Chris Livesey (right)

Paul GerrardLifetime Achievement Award

Pictured with Brindusa Axon

Barry Weston, Sogeti UK

Testing Innovator of the Year

Pictured with Mike Holcombe

MAIN AWARD OF THE

EVENING

SPECIAL AWARD

FOR MORE PHOTOS AND VIDEOS, VISIT WWW.SOFTWARETESTINGAWARDS.COM

TESTA ROUND-UP


Were you happy to be one of the sponsors?

Headline Sponsor "I think it was a tremendous success, recognising the excellent work which is going on in our industry. The tone was just right – a professional event with a fun atmosphere. "

“Sogeti had a super time at the first European Software Testing awards; the atmosphere was fantastic on the night. We are delighted to have won the Testing Innovator of the Year category and also proud to have received a shortlisting for Young Tester of the Year.”

“A fabulous evening celebrating success and talent in the testing industry; the organisation was flawless, well organised with a great atmosphere. It was a pleasure to be a sponsor.”

“It was great to see so much testing pride!”

“We found the event to be very well organised and Peter Dickson as a host was extremely entertaining. Our client, Marston’s Brewery, who we nominated as the client for Best Use of Tool award, enjoyed the evening immensely. Although we didn’t win, it was good to be part of the very first European Software Testing Awards and find out the latest within the industry.”

"The inaugural event was very well organised, and we are very happy with the award."

"The evening was a fantastic event and only served to highlight the breadth and depth of 'quality' within organisations and projects being delivered across Europe over the past 12 months."

How do you think the event went?

Headline Sponsor "We are delighted to be associated with TEST Magazine and TESTA. We look forward to an even bigger event next year!"

“Automation is something we pride ourselves on so we were really pleased to be able to sponsor the Best Test Automation Project category and, in turn, celebrate the great work of others in our industry.”

“BCS were proud to be a sponsor and to share the evening with our clients and the testing community. The calibre of the entries was high and demonstrated the value that excellence in testing brings to organisations and projects.”

“TestPlant gets its innovation from listening to testers so we’re really happy to have the chance to recognise the innovation of the industry.”

“We were delighted to be the sponsor of the Retail Test team of the year at The European Software Testing Awards because the event raises the profile of testing and celebrates excellence.”

"We are a major vendor in the testing space and sponsoring TESTA was a great way of giving back to the community we serve."

"The UKTB were delighted to be involved with TESTA which enabled us to further advance our aims of “professionalism” and “industrialisation” of the testing discipline across the UK and wider industry geographies."

WHAT OUR SPONSORS HAD TO SAY...

TESTA ROUND-UP


What would you like to say to the finalists and winners?

What do you believe TESTA gives to the industry?

Headline Sponsor "TESTA is offering a forum for professional networking; the exchanging of ideas; and exploring new and interesting solutions."

“At Sogeti, we believe that TESTA is a great platform to commend those in the software testing and QA industry and we are glad that, for the first time ever, there is now an awards ceremony focussed specifically on our industry. TESTA gives test providers and test teams alike a chance to review and promote their key skills, and it raises the awareness of the key players in the industry for anyone still searching for the right testing provider.”

“Recognising the achievement of the professionals working in software testing and software quality, and their essential contribution to the success of their organisations, is of paramount importance to the industry. TESTA gathered together the community for a prestigious event that acknowledged and celebrated success.”

“TESTA recognises the people who are really driving the testing industry forward with practical innovation rather than bombast.”

“We feel that TESTA will give more emphasis and credibility to the software testing industry.”

"TESTA recognises all the efforts and highlights the achievements done on a daily basis in the industry."

"TESTA recognises excellence is software testing and so encourages all software testers and companies to excel in the profession within which we all work. We’re very happy that TESTA has honoured a number of projects, individuals and companies at this year’s awards. We are looking forward to spreading the news wider across Europe in future years, as we all strive for global recognition of the software testing discipline."

Headline Sponsor "A hearty congratulations on the work you have done, and on the exciting recognition of being a true leader in the industry!"

“A big congratulations to all winners and finalists at TESTA! We feel proud to be counted among you.”

“BCS, The Chartered Institute for IT, applauds the achievement and professionalism of all of the finalists and winners. The contribution of these individuals and teams to the success of their organisations and their personal achievements is to be applauded; we are very proud to have shared this special event with you. Congratulations to all!”

“How are you going to beat that next year?”

“We would like to send a big congratulations to all the finalists and winners.”

"Keep up the great work."

"We were so pleased that a number of projects, individuals and companies have been recognised by TESTA this year. Additionally, we were thrilled that one of our mentors (Paul Gerrard) won the Lifetime Achievement Award. We want to extend to all those that entered our sincere thanks for their ongoing support of the industry within which we all work. Their efforts are always greatly appreciated."


TESTA ROUND-UP

ABOUT MAVERIC SYSTEMS

Started in 2000, Maveric Systems is a leading provider of Assurance Services across the technology adoption lifecycle, with a strong focus on the BFSI and Telecom sector. Maveric has built a business on the principles of independence, vertical focus and innovation. Today, Maveric partners clients from requirements to release with innovative IP-led solutions.

The company’s requirements assurance, application assurance and program assurance services are aimed at delivering successful outcomes on transformation programs for leading corporates in BFSI and telecom verticals. Maveric’s services are highly domain-led and this expertise is reflected in its superior solutions.

Founded with a focus on software testing in 2000 – the year the IT world was in a state of flux – when software testing was still at a nascent stage, and had not yet transformed into the booming sector it is today. Realising the growth potential offered by software testing as a service, Maveric decided to focus on domain-led testing as a dedicated line of business.

Over the years, Maveric has expanded its portfolio and is now geared up to provide integrated IT lifecycle assurance services. The journey here has been bumpy at times and smooth at others. Ranga Reddy, co-founder and CEO considers that the difficult phases have been important learning experiences and he, along with his team, has shown agility, focus and pragmatism in adapting business strategies to create a successful venture without compromising on long-term vision.

INCREASED RECOGNITION IN THE EUROPEAN SOFTWARE TESTING MARKETFinalists in The European Software Testing Awards’ Sogeti Best Automation Project and Testing Management Team of the Year award categories, Maveric Systems’ CEO,RangaReddy,shareshisexperienceofbecomingafinalist…

One of the most rewarding aspects of entering The European Software Testing Awards – or TESTA – is the process of reflecting on our successes and achievements over the past year. Whether this is as a team or individually, it meant reflecting upon the reasons for success; what we think makes our team and partners stand out. Entering the Awards is a testament to the successful domain specific technology transformation projects we completed and a reflection of the time and effort expended.

Thus, the Awards are a platform to celebrate innovation and be recognised for the effort put in by Maveric in pursuit

of delivering technological excellence. It is also a means to go head-to-

head with the world’s best testing and QA companies, and create

a benchmark as a leading integrated IT lifecycle assurance provider in the European market.

Being a finalist is a proud moment for us and it is a wonderful motivational boost for our team. It is a marker to the excellence delivered to our BFSI and telecom clients through that dedicated effort and focus by the entire team. Also it validates the growth voyage of Maveric over the last decade as a committed assurance partner. Enabling us to effortlessly partner our clients across the entire IT adoption lifecycle, from defining the requirement until the product goes live in the market and engage deeply with the client’s "business" in addition to IT.

We hope that being recognised as a finalist in the first ever TESTA will act as catalyst in creating increased recognition for Maveric’s lifecycle assurance approach in the European market. We would certainly encourage entries next year. The learnings are a great incentive! It’s a good platform to learn from the experiences of the best of technology companies.

BEING A FINALIST IS A

PROUD MOMENT FOR US AND IT IS A WONDERFUL MOTIVATIONAL

BOOST FOR OUR TEAM


TESTA ROUND-UP

Sophie: How does it feel to win?Brickendon Consulting: Winning The Sogeti Best Automation Project is great formal recognition within the industry for Brickendon Consulting’s skills. It is the perfect payback for our efforts and will help inspire us to continue to develop new products in the future.

SO: What do you think it will do for your profile?BC: We hope it will further raise Brickendon Consulting’s profile and show potential clients what we are capable of, and that we are good at it.

Even though Brickendon Consulting has been around a short amount of time, within the industry we are already working with the best of clients. We are playing with the big boys.

SO: Why did you enter the awards?BC: The awards offer a great opportunity to gain recognition for our solutions and services. As well as showing how we can help our clients save time and money.

Achieving finalist positions for three awards (Best Use of Tools; Testing Innovator of the Year; and Sogeti best Automation project), was great for morale within Brickendon Consulting and has shown us that it doesn’t matter what size your company is, you can still come up with strong, innovative solutions.

It has raised our level of awareness within the industry and given us confidence to pitch against bigger consultancies. It has helped the whole of the Brickendon team, encouraging new people to join the company and giving us some formal recognition for our efforts.

It has given Brickendon Consulting a way of showcasing our talents and helped cement a culture within Brickendon Consulting of driving new solutions based on our clients’ and consultants’ experiences.

SO: Why would you encourage others to enter next year?BC: The Awards were a perfect way to promote Brickendon Consulting’s work. It showcased credibility, gave confidence and offered some recognition for our efforts.

SO: How did Brickendon start?BC: Brickendon Consulting was established in May 2010 in response to a cry for help from a client. From small beginnings with just four people on site at that client, Brickendon Consulting has grown to a 65-plus strong team based at the group’s head office in the City and on site at financial clients around the capital.

SO: How was it developed?BC: With two core platforms of Management Consultancy and Technology Consulting, Brickendon Consulting is going from strength to strength with more than 30 projects in a variety of investment banks and energy trading firms.All of our consultants have an average of 10 years’ experience within the industry, and are highly qualified to address common challenges faced by clients. As a consultancy we have developed the Brickendon Approach, which allows for a structured solution to problems in areas such as eTrading, regulatory reform, trade monitoring, business change, energy trading and test and quality assurance.

We are passionate about innovation and seek to drive change through the delivery of complex business and technical solutions. Through our Knowledge Leadership function, we are incorporating targeted research and knowledge management, with the aim of showcasing the skills and experience of our consultants through case studies and solutions documents. These can also be used to share challenges and help develop new ideas to promote Brickendon Consulting’s original thinking and dedicated approach to clients’ problems. To learn more go to www.brickendon.com

We are continually winning new business and are on target to achieve our goal of growing to 200 consultants by the end of 2015.

"THIS IS FORMAL RECOGNITION FOR OUR SKILLS"Winners of the Sogeti Best Automation Project, Brickendon Consulting, share what this award win means...

IT DOESN’T MATTER

WHAT SIZE YOUR COMPANY IS, YOU CAN STILL COME UP WITH STRONG,

INNOVATIVE SOLUTIONS

DECEMBER 2013 | WWW.TESTMAGAZINE.CO.UKPAGE 30

TESTA ROUND-UP

Tech Mahindra explains how winning the BSC Best Overall Project award and the award for Best Use of Tools was the perfect end to an already successful year, and solidifies that they’re at the forefront of innovation and service delivery in test and quality assurance services, across the European testing marketplace…

Tech Mahindra was proud to win two awards, including the category for The BCS Best Overall Project and Best Use of Tools at The European Software Testing Awards (TESTA), which took place in London on 20 November. TESTA provides a great platform to recognise expertise across the software testing industry, with experienced judging of entries, and the company was very proud to receive these accolades, which are judged and awarded by peers in the software testing industry.

Tech Mahindra’s Global Test Factory was created two years ago to provide an alternate solution to traditional testing contracts. The approach uses a layer-based operating model and focuses on velocity, standardisation, quality, consistency and low running costs.

Investment in bespoke tools as part of a combined strategy with off-the-shelf utilities proved a key aspect in winning the category for Best Use of Tools.

In the winning “Overall” project, 50 plus industry best and bespoke tools were developed and deployed over the course of the contract to date. These provided significant benefits (with a time saving of up to 97.5% in specific cases) across innovation areas such as early engagement (shift left); depth of testing; demand (forecast and capacity); test management; fail fast and defects; diagnostics; and operation assurance.

TESTA provided further industry recognition of the success, and comments from the judges recognised Tech Mahindra’s commitment to high quality and standards. Commenting on their Best Use of Tools award win, judges said that Tech Mahindra had successfully implemented a combined approach, utilising bespoke applications, COTS applications and open-source tooling with in-house developments. The project had clear aims, a robust schedule and a clear mandate to deliver improvements.

In regards to The BCS Best Overall Project, judges felt that Tech Mahindra showed a great commitment to high quality and standards. And their efforts to keep up-to-date with guidelines and trends, and dedication to self-training were commendable.

Sophie-Marie Odum, Chair of TESTA judging panel, added, “Congratulations to Tech Mahindra on these fantastic achievements. They were truly well-deserved as judges and I were very impressed with these entries. They have certainly set the standard for next year.”

The Test Factory is seen to deliver added value to customers and partners, and establish a capability a step above managed services. It also offers a full range of scalable testing services, supported by a strong process and performance driven culture.

Earlier in 2013, the Tech Mahindra Global Test Factory was evaluated and certified at TMMi level 5. This certification had an additional distinction, as Tech Mahindra became the first company in the world to achieve a TMMi level 5 rating at the first attempt.

Achieving the highest level of industry certification confirms Tech Mahindra’s place at the forefront of innovation and service delivery in test and quality assurance services, across the global testing marketplace… as does this double award win!

JUDGES FELT THAT

TECH MAHINDRA SHOWED A GREAT COMMITMENT TO

HIGH QUALITY AND STANDARDS

DOUBLE AWARD WIN CONFIRMS COMPANY’S PLACE AT THE FOREFRONT OF INNOVATION AND SERVICE DELIVERYTech Mahindra explains how winning the BCS Best Overall Project award and the award for Best Use of Tools was the perfect end to an already successful year, and solidifiesthatthey’reattheforefrontofinnovationandservicedeliveryintestandquality assurance services, across the European testing marketplace…

As our world connects itself, we embrace a new identity in the form of our brand and purpose - inspired by the past and motivated by the future.

Tech Mahindra's Global Test Factory is the next generation concept in soft-ware testing, providing a superior and flexible alternative to standard off-shoring model. It is a multi-location based testing center of excellence that combines best-in-class test processes, people, techniques, facilities and methodologies. It also offers a full range of scalable testing services, cutting-edge innovation along with optimum unit cost of testing..

Welcome to a connected world of connected solutions to connect better with your customers, improve your customer experience and opti-mize operations.

Connect with us:http://www.techmahindra.com or write to [email protected]

TESTING & PERFORMANCE.CONNECTED.


TEST DATA

oday’s enterprise applications are designed and developed to encompass several interrelated sub-systems, which are hosted on multiple heterogeneous technology platforms. So, when one of these applications undergo a

change, or when a new sub-system is added to the existing applications suite, extensive validation and end-to-end testing of few important business processes is required.

To realise this, a new functional verification environment needs to be created and then the interrelated applications will be made available. In such situations, the testing team need to first address the requirements with two important stages of the test data management lifecycle. They are t preparing the est data and managing the on-going test data. However, the team may encounter a few challenges while working through both the stages.

TEST DATA PREPARATION CHALLENGESHere are the challenges associated with test data preparation activity:

• Managing contemporaneous data requirements in multiple test environments.

• Formulating test data to suit non-production environments.• Ensuring appropriate test data for all applications across

all environments. • Security and compliance restrictions for using

production data as is for testing.• Ensuring privacy of the data by not disclosing

sensitive information.

However, all the aforementioned challenges can be readily addressed as there are several tools, techniques and best practices available. Importantly, to readily counteract some challenges, the team may implement following strategies like:

• Implementing some utilities and appropriate process for data extraction and loading.

• Make use of some data masking tools.• Use data sub-setting techniques. • Synthetic creation of new data for testing, etc.

By making use of these strategies/techniques, the testing

teams will attain a thorough understanding of the test data requirements and will be able to provide an appropriate solution during the test data formulation phase.

Similarly, administering the on-going test data activity is associated with the following challenges:

• Administering reference data: It is essential to have well-organised reference data, which can be used as a coherent source of refreshing the test environments whenever there is a need.

• Ensuring test data quality in the test environments: It is very important to ensure test data quality in the test environments. This is because there is every possibility that some inconsistencies may creep into the test environment as testing progresses due to several factors, such as repeated execution of negative scenarios, application related errors, etc. So, during such situations, making use of some data quality analysis techniques would certainly help to disclose these issues in time and retain the data quality.

• Rationalising data provisioning process: The data provisioning process can be improved and/or streamlined by ensuring right and specific data for testing complex scenarios, and considering inconsistent data request formats and unclear data requirements; lack of completeness of information between data and test scenarios; test data depletion; ineffective reuse of existing test data and assets; and above all, lack of proper understanding and visibility with test data management activities.

MANAGING TEST DATA – AN ADMISSIBLE APPROACHHaving experienced the challenges, it calls for an idealistic approach which could reduce the test data management effort considerably by ensuring a smooth, integrated and well-defined workflow. This approach would typically comprise of a few generally admissible and effective steps, namely:

• Maintaining reference data This forms a prerequisite for any test data management. It may not be a good idea to use and follow some test data preparation activities to extract data as and when required. Rather, a reference repository of test data

LINKING TEST DATA MANAGEMENT AND QUALITY SOFTWARE — PART 2In August, Srikanth Dora Karam and Narayana Maruvada discussed how affordable test datamanagementsolutionscansupporttestefficiencyandstreamlineefforts.Inthisissue, they explain the challenges with test data preparation and offer a solution…

T

SRIKANTH DORA KARAMTECHNICAL LEADVALUELABS

NARAYANA MARUVADASYSTEM ANALYST - QAVALUELABS


TEST DATA

should be maintained. So, when there is a need, test environments can be refreshed using the reference data. Later, required data can then be extracted from the production environments after applying appropriate data masking methods.

• Conducting data quality analysis at regular intervals

Conducting periodic data quality analysis of test environments is very essential as it helps ensure a high quality of test data is available for effective application validation.

• Implementing a comprehensive data provisioning system(workflowbased)

A data provisioning system provides a “common” for placing the test data requests and data management activities. This ensures data request management, mapping data requests to test cases and reservation of data.

BEST PRACTICES CONCEPTUALISEDA good test data management process should adhere to the following five best practices for ensuring an effective, systematic and result-oriented way of handling the end-to-end test data related activities:

1. Creating copies of production data on test environments

This is often referred to as migration of production data to test environments, and it is essential practice. It helps in defining the different test environments in which the production data has to be cloned or migrated. The size of the test environments is generally determined by the purpose and usage of those environments. This step would typically involve the process of making a decision of how much production data needs to be replicated into which test environment.

2. Creating supplementary data on test environments Generally, new test data may have to be created beyond the migrated production data based on the type of test environment and the type of testing required like

functional, performance and data warehouse etc. This in-turn depends on a) types of tools and techniques used for data creation, and b)

exceptions / miscellaneous reasons under which projects can create new

data, etc.

3. Cleaning up test environments post-test completion

This is another essential practice whereby, in order to test all needed scenarios, test data may have to be modified which leads to the existence of altered test data at the end of the project testing. So, a clear methodology or approach should be in place with some directives detailing how and when the cleaning up of the test data needs to be done after the testing completion. However, taking certain guidelines into consideration, the altered test data cannot be, or should not be cleaned up.

4. Data confidentiality and protectionOnce it is decided by the team that data needs to be cloned, first all questions and concerns related to data privacy need to be addressed. All the tools, techniques, mechanism and stages of cloning process, at which depersonalisation would be implemented, should be clearly stated and defined.

Importantly, if there is a situation where the testing being outsourced or offshored, then it is very essential that the organisation along with service provider jointly has to decide on the data privacy mechanism and work accordingly.

5. Managing test data requirements of on-going projectsThere will be one definite situation where the respective project teams would struggle with conflicting project priorities for the test environment. So, to avoid them, a clear guideline and mechanism should be in place to prioritise and decide which project should go for the testing first and make use of the test data and environments.

Following and integrating these five best practice guidelines can ensure a good test data management process and ensure an effective method for handling end-to-end test data related activities.

HAVING EXPERIENCED

THE CHALLENGES, IT CALLS FOR AN

IDEALISTIC APPROACH WHICH COULD REDUCE THE TEST DATA MANAGEMENT EFFORT CONSIDERABLY BY

ENSURING A SMOOTH, INTEGRATED AND WELL-

DEFINED WORK-FLOW

built by

THE QUEEN’S AWARDSFOR ENTERPRISE

INTERNATIONAL TRADE2013

Software quality toolsfrom design to production

Easy-to-use tools which empower designers, developers, testers and managers toreduce time to market improve quality save money

from mainframe to mobile


SECURITY

DO BUSINESSES UNDERSTAND THE IMPORTANCE OF PENETRATION TESTING?John Yeo, EMEA director at Trustwave, who heads up the ethical hacking division, SpiderLabs, speaks to Sophie-Marie Odum about how today’s constantly evolving security threats mean mere point-in-time testing could unknowingly leave a network or application exposed to an attacker and businesses could possibly face a costly, damaging attack…

Sophie: Why are businesses finding that they need to constantly update security plans?John: Many businesses are continuously changing and updating their technologies. As such, this can mean their security activities have to change, adapt and evolve their plans.

SO: What kind of security threats do you think businesses face?JY: Businesses face a wide variety of security threats. In 2012, Trustwave conducted more than 450 incident response investigations and determined that the vast majority of data breaches involved the theft of personal data records from businesses.

To gain unauthorised access to personal data, attackers invariably exploit vulnerabilities in web applications and/or Internet-facing infrastructures. That’s the first step in gaining a foot-hold into the target environment; a first step towards stealing data.

SO: Please explain the importance of penetration testing to businessesJY: An organisation typically invests a percentage of its budget on security, and one of the overarching goals of a robust security strategy is to proactively identify weaknesses in a business’s security before criminals find those weaknesses.

Companies should have security controls in place that restrict inbound access to a network such as a Web Application Firewall and Network Access Control solution, as well as technologies that identify unusual activity, however, in addition, to those kinds of security technologies, businesses should also perform frequent penetration testing to help identify any vulnerabilities in their security so that they can fix those vulnerabilities before it’s too late.

Penetration testing helps verify they have the proper security controls in place to protect their information and helps businesses determine if they need any additional security technologies.

SO: Whose role is it to initiate this within the business?JY: A business should understand who is responsible for security and it should be a well-defined role with clear

responsibilities, with a clear mandate for improving security and addressing information security risks.Furthermore, the role should have support from the executives within that business. Security managers should not operate on an island, isolated from the rest of the company. Executive support is crucial to addressing information security issues across various departments and making security a top priority within the company.

SO: What are the consequences of not testing?JY: The goal of penetration testing (or security testing/ethical hacking) is to determine whether it is possible for a malicious user to subvert the security controls in that application (or other IT asset) in order to induce it to perform functions that it was not intended to do.

A penetration test will determine what real-world scenarios would actually be possible if a “bad actor” made concerted attempts to cause harm to either the business, or its customers, through malicious activity. It is demonstrative and will identify how and where someone could gain unauthorised access to systems and data. Through a penetration tester’s skills and experience, it will be possible to demonstrably circumvent security measures.

Not doing so would be similar to making assumptions and not attempting to validate those assumptions, i.e it is possible that production applications or infrastructure would be unable to withstand an attack if targeted by a malicious hacker.

SO: In your opinion, do you think enough companies conduct such testing? If so, how often?JY: Businesses should have penetration testing performed frequently throughout the year, so that as more technologies are added to the corporate environment and threats become more complex, businesses are continuously identifying and fixing weaknesses in their security and staying a step ahead of the criminals. This is the most effective way to stay ahead of the latest threats.

However, some companies may believe that because a vulnerability has not yet been detected or exploited, it is unlikely to happen in the future. They also may not realise that regular penetration testing can actually improve non-functional aspects of software (i.e not understanding that a software’s security properties are an important component of software quality).

JOHN YEO EMEA DIRECTOR TRUSTWAVE

DECEMBER 2013 | WWW.TESTMAGAZINE.CO.UK PAGE 37

WATCHING ME, WATCHING YOUBrickendon Consulting explains the importance of security testing...

ecurity is something that matters to us all. Whether you’re Barack Obama coming under fire for flaws in his healthcare website; a LinkedIn user concerned about the confidentiality of personal dat;, or the head of

a City bank seeking to ensure the safety of their financial and trading systems, it matters.

Earlier this month, a cyber attack by a foreign government on financial markets was played out in one of London’s historic halls with the aim of testing the City’s defences against online saboteurs.

About 100 bankers, regulators, government officials and market infrastructure providers took part amid concerns about the threat of cyber crime to the banking system, including the impact of coordinated online assaults or hacking attacks on specific lenders.

While this may sound rather dramatic, when it comes to online security you can’t be too careful.

Where most of us spend our evenings relaxing on the sofa watching TV, there are others, known as "white" hackers who are preoccupied with other matters. Their raison d’etre is to find faults in other people’s software and technology systems.

“That is what they do,” said Viswa Muthukrishnan, a specialist in testing at Brickendon Consulting. “They play around for hours with the aim of finding loopholes or vulnerabilities in existing systems. Once they’ve found something they then report it to the website’s owner or developer.

“If the in-house team find that there is in fact an issue, then the hacker is paid for his services.”

This is obviously a positive outcome for the company behind the system or the individuals with data stored on that site, with a "white" rather than conventional hacker discovering the fault. Unfortunately, things don’t always work out like that, and security breaches to websites, online payment systems or data storage centres can have significant consequences for both individuals and businesses.

As an individual you want to be sure any details you input to a banking or payment service are safe, while as a company, your reputation, and in turn profits, are at risk if you fail to offer clients a secure environment in which to operate.

While companies are becoming increasingly aware of their obligations when it comes to security and the impact it can have on their business – indeed, according to a study by one London-based security firm, 52% of the world’s 50 largest banks have been impacted by a software security breach – the lack of uniform global regulation means that there are no set ways to address the issue.

Security experts at Brickendon Consulting suggest that security testing should be approached in the same way as functional testing.

“It could save businesses significant amounts of time and money if you implement security testing at the same time as operational testing,” said Muthukrishnan. “That way you are using the same resources to address two issues simultaneously.”

By doing this you are also introducing security testing into an accepted structure and already-defined business process.

“You would never let a piece of software go live without testing its functionality, so why let it be used before its security has been properly tested?” said Muthukrishnan.

The other issue is that technology, which is found in all areas of business and in all functions, is continually evolving. As a result, there are plenty of opportunities for rogue or unfit for purpose software to slip under the radar.

So, with Americans worrying about the security of their government’s healthcare website and British Defence Secretary, Philip Hammond encouraging the military to build not only cyber defences, but also an offensive cyber capability, it is no time to be complacent.

Do you know the last time someone hacked into your website or financial software? Have you thought about what a security breach would do to your business’ reputation, and what your reputation means to the future of your business?

IT COULD

SAVE BUSINESSES SIGNIFICANT

AMOUNTS OF TIME AND MONEY IF YOU IMPLEMENT SECURITY

TESTING AT THE SAME TIME AS OPERATIONAL

TESTING

S

THOUGHT LEADERSHIP

PAGE 39

TEST AUTOMATION

DECEMBER 2013 | www.testmagazine.co.uk

CROSS-BROWSER FRIENDLY WEBSITESPrashant Chambakara discusses the nuance of cross-browser testing and conducts a comprehensive analysis of how it should be conducted…

hen a web or cloud-based application is released, what concerns the developing company most is its compatibility across various browsers. A slight lag in the look or working of an application, on any browser,

diminishes months and years of hard work and lowers the company’s reputation.

With a growing number of browsers, it’s imperative that an application is compatible with all, which is why experts have diagnosed and prescribed the need of cross-browser testing.

WHAT IS CROSS-BROWSER TESTING?

Cross-browser testing is generally adopted to ensure that a website or web application works properly on different browsers. Ideally, it is the testing of a website, web application, HTML file or client-side scripting to support all the web-browsers. Essentially, it involves a compatibility check and testing of both client-side as well as server-side applications.

With the increase in website and web application access across smartphones, tablets, laptops and iPads, which work on different browsers (e.g. iPad and Mac devices use Safari while Android tablets and smartphones either use Firefox or Chrome), cross-browser testing becomes inevitable in order to present a unified user experience and ideal functionality.

In addition, cross-browser testing benefits in several aspects such as:

• Simplifying website management and updates.• Rendering consistent functionality across all the

web pages.• Providing ease of viewing to end users.• Increasing the customer conversion ratio.

WHICH BROWSERS SHOULD BE TESTED FOR COMPATIBILITY?There are several browsers that are used; however the most used browsers across the world include Google Chrome, Firefox, Internet Explorer, Safari and Opera.

To help prioritise web application and website testing across these browsers, here are our statistics of each ones usage across the world:

• Chrome: 52.8%• Firefox: 28.9%• Internet Explorer: 11.8%• Safari: 3.6%• Opera 1.6%

WHAT TO TEST FOR BROWSER COMPATIBILITYThis question concerns many. For checking whether the application works the way it should across all the browsers, you should take a look at the following ingredients of a web application or website:

• HTMLHTML is the main component of website pages hence why it is the primary ingredient that should be checked for compatibility.

• CSS styles and CSS validationsIt’s essential to check CSS styles and CSS validation in verify to check the compatibility since CSS has moved to CSS3.

• Sessions and cookiesIt is also necessary to check sessions and cookies of each website page.

• JavaScriptJavaScript also requires attention as it varies from browser to browser.

• Text alignments and page layoutsWhile testing cross-browser compatibility of web apps, you must also check the dropdown; checklists; radio buttons; and front-end UI for each browser as they may not be similarly aligned in all the browsers.

Over and above these, there are several other important areas of web applications to be considered for achieving 100% cross-browser compatibility, such as:

• Font sizes• Mouse hover• SSL Certificates• Upload file• Export file• Scroll bar appearance• Flash work• Page zoom in/out• Pop ups• Space between various HTML components, etc.

When you are unaware of what browser is being used, and there is such a wide choice of browsers for running your application, checking your application on each can turn out to be a big headache, and, if not done correctly, can leave you vulnerable. However, there are some automation tools that are specifically designed to test the application performance and work with different browsers to minimise the testing efforts.

When it’s time to make a website or web application completely functional from the mock-up stage, it’s crucial to ensure it works great on every browser, meeting client expectations and, more importantly, allowing end-users to use the website or application with complete ease.

W

PAGE 41

TEST AUTOMATION

DECEMBER 2013 | www.testmagazine.co.uk

WHAT’S IN A GLITCH?Automation is the key for businesses preparing and testing against worst-case scenarios. Neil Kinson, VP EMEA at Redwood Software, discusses the importance of anticipation when it comes to technology testing…

he banking sector is no stranger to IT glitches. But a string of recent incidents raise questions about just how seamless its IT automation processes really are. Goldman Sachs recently missed a multibillion-dollar order at an auction

of Treasury bills as a result of a “glitch” in its computer system. NASDAQ also recently experienced a technology glitch which meant that investors could not view their stock quotes for a whole six minutes.

Instances at both of these organisations were cited in a recent Standard & Poor’s report, which warned that issues like this could put their ratings at risk. The report highlights the tangible and lasting damage that result from technology mishaps. If failures like this continue to chip away at market confidence, borrowing costs could rise, leaving exchanges at a real disadvantage – it’s far more than just reputations on the line here.

However, there is a fundamental problem with the use of the term “glitch”. Rather than technological failures, it is often human error which leads to incidents such as the ones at NASDAQ and Goldman Sachs. The mistakes commonly begin in the data collection stage or in the operation and testing of the system itself. If experience has taught us anything, the common component in almost all incidents like these is a forgotten need for risk assessment and thorough evaluation. If businesses make the necessary preparations against worst-case scenarios, they reduce their chances of falling victim to such problems.

USE THE “AUTOMATION” LIGHTCompanies can avoid and resolve a plethora of manual errors with an enterprise process automation approach. Good automation keeps a handle on the problems that the IT team doesn’t have time to manage or worry about. However, as with all technology; automation is only as good as the information in its foundations.

One of the keys to clever automation is notification. IT teams, working with very high-specification environments tend to rely on their systems to automatically failover in the wake of an IT problem. When preparing for such pitfalls, the IT department needs to locate at what point, and how, human intervention should happen.

What these recent incidents have in common is that the point of human intervention was either not identified or adequately planned for. In each instance this caused significant failure, as the technology didn’t operate as intended when the parameters around it changed. Any successful IT department needs to understand all the likely scenarios that can affect their system and they should preemptively test against them. This can be done with the help of automated notifications.

The most likely case of technical "glitches", such as the one affecting NASDAQ, is as simple as an unanticipated amount of volume transactions. The network had always coped under normal volumes, so it is possible that nobody had predicted, or, more importantly, tested how increased loads would affect the performance. No boundaries had been set to alert the team that something was going awry. That left the IT department essentially operating a printer without a “check ink levels” alert. The first time the team knew anything was wrong was when the printer simply didn’t print.

IT teams should be aware of the saying, “Anything that can go wrong, will” because expecting and preparing for IT glitches makes for more effective solutions when things do actually go wrong. This is where automation should help IT teams diagnose and correct errors before they become a real problem. At the application level, this can speed recovery from any issue. Nevertheless, it requires persistent testing to prevent a failure from happening in a real-world scenario.

NOTHING VENTURED, NOTHING GAINEDNetflix is a prime example of this sort of approach. It uses an automated problem-creating system called "Chaos Monkey" in its network. This application is programmed to cause failures by seeking out Auto Scaling Groups (ASGs) and terminating instances (virtual machines) in each group. It does this during certain hours and days when the IT team can respond quickly to the outages.

By organisations causing their own system problems, the team can preemptively resolve them, taking action against their own, imitation outages before a legitimate one takes them by surprise. By forcing ad-hoc failures across the network, the IT department builds in resilience against outages without compromising on performance. This means the company stands a better chance of not falling victim to a genuine failure that could result in real business losses. This kind of testing helps safeguard against issues that come from real-world complexity, which can often build up from small issues.

There is seldom one stand-alone reason for a failure of the scale of the recent NASDAQ or Goldman Sachs incidents. Situations like this are often the result of a collection of smaller issues that build the perfect, unforeseen – but completely preventable – storm. To ensure that businesses make the most of the opportunity ahead, they need to scrutinise the worst-case scenario in order to understand how to best improve their business. Let automation deal with the small things and inform humans to take the wheel when needed. Collaboration is the key to keeping everything running smoothly between tightly automated processes and insight, intelligence and human insight.

T

NEIL KINSON VP EMEA REDWOOD SOFTWARE

References available on request


THOUGHT LEADERSHIP

STEVE MOORE TECHNICAL AUTHOR

MICRO FOCUS

IS IT RIP FOR QA?Steve Moore, technical author, Micro Focus, discusses whether QA is dead, or if a different approach is required…

he apparent demise of QA has been likened to the early days of the Industrial Revolution, when “new” technology overtook the efforts of simple labour. If that’s the case, then what is the equivalent of the steam engine; what will

happen to the honest artisans whose bug-spotting roles are – apparently – going the way of the farm labourer? And where does that leave the software industry?

So is it dead, or does it just suffer from poor perception and bad practice? Well, to paint with broad strokes for a moment, it depends on who you ask – and what you define as QA. To understand what traditional QA is, it’s helpful to define what it isn’t.

QA vs QCFor one thing, it is not a verb. Software is not “QA’d” as such, but the assets are verified, from an end-user

perspective to ensure they work properly. The traditional QA tester will use either a waterfall or agile approach to assess software build quality using a pre-defined approach.

If we accept that waterfall is a top-down, requirements-driven approach with quality at the end, while agile involves quick iterations with quality “baked in”, then the process is defined and we have a time-proven – and time-proof – way of ensuring that software is fit-for-purpose and good to go. Except of course that we don’t, do we?

If we agree that Quality Assurance is process, and testing is Quality Control, then the two must be totally aligned to be effective. QA ensures the process is (a) sound, and (b) does what it should. QC determines whether the product meets customer specifications. Clearly, both are needed, but if the product doesn’t perform, then the question is about whether the problem is a product (ie technical) or process (ie invalid requirement) issue? So is it QA or QC? Does that sound totally aligned?

T


THOUGHT LEADERSHIP

FASHION-CONSCIOUS It’s difficult to pick up an industry magazine or scan a testing website without seeing a discussion on mobility, the current testing trend. It’s big, but at this stage it’s difficult to see if it’s sustainable. According to Forrester, while 83% of customers have invested in a mobile strategy, fewer than half of them have a mobile roadmap for the next 12 months.

Clearly the longer trend is towards faster, more rapid development, which suggests an agile approach. But who knows how long even this will last? Remember when waterfall and then agile were the next “big thing" in software testing? Some embraced both, or just one of these approaches. Others fused the two to create a testing hybrid. Agile and waterfall are very different methodologies with one thing in common – they were both the next big thing that was going to revolutionise how we deliver software. So what happened?

Well, we think that the confluence of the five different elements listed below means that yes, QA as we knew it is dead – but the process has simply evolved quicker than either of the current definitions.

Let’s take a look at the five factors driving the change in definition:

1. Tablets, mobile and other devices: The proliferation of “the next thing” is constantly evolving. Most new development will happen on these platforms to substantiate the growth of ubiquitous computing.

Bring Your Own Device (BYOD) isn’t going back in the bottle and raises its own challenges for IT. Whether organisations choose to implement it or not, employees as IT consumers see computing as a single, transferable entity that works away from the desktop, as something that happens everywhere and anywhere, on any device, in any location and in any format.

To be successful, testing must be performed early and often and work for the underlying technologies – that is the code, middleware and operating systems – behind ubiquitous computing.

Current QA assets being used now, from scripts to unit tests, must expand to include new machines, devices and frameworks. Use cases will need to be rigorously simulated meaning more visual and user-driven tests. This pushes the boundaries of the current definition of agile and will surely fuel the growing interest in Continuous Integration Testing (CIT).

2. Consumer expectation: The genie is well and truly out of the bottle. Consumers demand more than ever and

expect apps run on any device. So QA is now more than just validating features

and functionality – everything’s geared towards customer satisfaction. QA must

ensure quality – SDLC tools can import and track feeds from many social sources, spend less

time on features and more on consumer ideas.

3. HTML5/hybrid apps: A widely-quoted Gartner prediction suggests that by 2015, HTML5 will deliver 50% of the applications that would have been native in 2011. There are pros and cons to this freedom. While developers can leverage common web skills and build cross-device apps quicker and cheaper than ever, doing so without access to all the native device capabilities can compromise the end-user experience.

Tests must also be run through a variety of device types and/or emulators to verify that changes and response times are accurate. Testers must ensure items like GPS, accelerometer, position data are all accurate or disabled when necessary. It’s another testing QA challenge.

4. Software supply chain: Delivery and test collaboration is crucial in today’s SDLC but market dynamics are such that the results are driven by cost and complexity. Testers are finding confusion and delay. Inflexibility in scheduling and the ability to quickly change direction belies the “agile” tag.

To succeed, QA, requirements resources and development frameworks must all be flexible, and every SDLC asset fit-for-purpose – linking adaptive requirements and test acceptance to the developer’s IDE is a good start. If not, then QA and requirements will continue to hold up the process and act as a lightning rod to "QA is dead" arguments.

5. Delivery guidance: As more devices and apps go online, issues are no longer limited to localised code and DB calls. They’re more about availability, performance and latency. So QA must evolve to find the resolutions that meet these needs. QA teams will need to understand how delivery mechanisms work in future architectures because only QA – or an offshoot of QA – can ensure the additional complexity works where and when it should.

More pertinently, the business must understand that consumers now drive the market and developers and testers are left, to some extent, to second-guess what is happening next. Consumers drive usage and the direction that manufacturers take in creating devices. It’s certain that there will be more “points of access” to come and the future will remain mobile. The industry’s attitude and approach to testing will just have to go with it.

The bottom line is that processes are a valuable framework to organise and perform all other project tasks.

CLEARLY THE LONGER

TREND IS TOWARDS FASTER, MORE RAPID

DEVELOPMENT, WHICH SUGGESTS AN AGILE

APPROACH. BUT WHO KNOWS HOW LONG EVEN THIS WILL

LAST? REMEMBER WHEN WATERFALL AND THEN AGILE

WERE THE NEXT “BIG THING" IN SOFTWARE

TESTING?


Within that framework processes can be improved, they can be shared and they can be passed on. In other words, they evolve beyond current definitions.

...AND THEN THERE’S SQA

Software Quality Assurance (SQA) polices the police. It monitors the quality of the processes that monitor the quality of the product. Whether the SQA process conforms to a standard, such as ISO 9000 or a model, such as CMM or TMMi, it’s the all-encompassing process that goes from design right through to delivery and product integration. It achieves this through the setting of goals and the meeting of commitments. It draws on abilities, creates activities, takes measurements, and delivers verifications.

Unfortunately it’s a misunderstood or incorrectly deployed role. SQA can feel like software testing’s “thought police”, forever investigating, auditing and regulating projects and those charged with delivering them. A good idea often badly implemented, SQA groups are reported to have a life expectancy of about two years. It’s short because SQA is seen as a deconstruction of other people’s work rather than creation of a solution in its own right, making a business case for perpetual funding difficult to make.

HOW “ASSURED” IS YOUR QA?

In reality – probably not very. How could it be? Successful development, even within the context of a formal software life cycle, requires everyone – developers, BAs, testers, architects, DBAs, trainers and management – to get everything right every time. Guaranteeing perfection is like holding a hostage to hubris. It’s impossible to achieve it without first testing everything using a process that may be in itself flawed.

Because every line of code is part of a bigger structure which could have thousands of possible inputs, outputs, states or dependencies. Each one must be regarded as a potential point of failure, so it must be tested. But how? Even documenting every test case doesn’t factor in impacts from other lines of code or external factors – and addressing all these is a near-impossible task.

THIS ISN’T EASYEven a seemingly simple requirement, such as supporting multiple browsers, exponentially increases the difficulty of building and testing software. A common requirement is to support multiple versions of each browser, so the complexity and difficulty increases again.

The talk around the developers’ water-coolers is that software development is hard because programming is

easy. Because there is little in the way of professional accreditation, mistakes from inexperienced and/or under-skilled programmers are adversely affecting projects. This view lays bare the differences between testing and development. Testers can only find

the bugs they think to look for and if development and testing teams aren’t

connected, organisations won’t create the robust code that makes great software

products and happy customers.

CAN WE FIX IT? WELL, MAYBE...It’s a given that the software building process will generate bugs. These need finding and fixing. Without this effective gatekeeping, flawed products are inevitable. Bad QA compromises all the good stuff – the functionality and features – the developers spent so long creating and refining.

So the emphasis must be on effective QA, right? Well, if there’s time. The perception is that rigorous QA is an optional extra. If the development phase overruns, the testing deadline remains fixed giving teams less time and – perhaps – a sense of resentment that testers and their craft do not get enough recognition.

QA in some form is essential, and it can work well. The problem is where there’s a disconnection between the QA approach to the business and project context, where the process is so templated it isn’t adaptable enough to match the individual requirements of the project or the product. QA is necessary but each organisation must decide how best to adopt it. Some organisations regard it as a safety net, rather than generating a system that enables a team-based “go live” decision.

Surely there’s a better way to deliver QA? A mechanism that ensures the testers are helping to create the product that the customer wants? Can organisations really expect to perform 100% of their testing – and if they do, how will they prioritise this workload? And while they are coping just fine at the moment, the added complexity that the move to mobile – both the devices we know about and those that we have yet to see – will add increasing complexity to these processes. How will they cope then?

The business stakeholders are probably the only people who can match the level of software quality with what the end-user – the business customer – will experience. Implementation of real-time monitors, linking social CRMs to the SDLC infrastructure, and/or deploying a performance and load testing solution with full diagnostics would ensure that thresholds and latency times are within acceptable thresholds.

THOUGHT LEADERSHIP

GUARANTEEING PERFECTION IS

LIKE HOLDING A HOSTAGE TO HUBRIS. IT’S IMPOSSIBLE TO ACHIEVE IT WITHOUT FIRST TESTING

EVERYTHING USING A PROCESS THAT MAY BE

IN ITSELF FLAWED


As a mix, this would enable the visibility and help drive the creation of an infrastructure that would identify the issues and help drive the resolution.

So, "is QA dead?" It’s an eye-catching statement that delivers more than 20M Google results. But if you take out the vested interests – i.e those who want it to be so, rather than those who believe it to be so – then you’re left to make up your own mind.

It’s certainly true that QA, rather than just testing, suffers from bad practices, perhaps as a result of not moving with the increased demand for additional features and functionality. There’s certainly an image problem and a feeling that technology is moving faster than our way of testing it.

Let’s be fair: the software industry is only about 50 years old so any

“traditional” activity isn’t so carved in stone that it can’t be restructured to

factor in new products, people, systems – and testing products. When the car was

first invented, the “process” included the legal requirement for all self-propelled vehicles to be led

by a pedestrian carrying a red flag. That seemed sensible then, but technology has a way of quickly rendering current thinking obsolete.

Whatever your view, it’s clear that, like the Industrial Revolution, change can happen quickly and those who accept it, embrace it and work around it will leave those who don’t behind like an express train overtaking Stephenson’s Rocket.

THOUGHT LEADERSHIP

LET’S BE FAIR: THE SOFTWARE INDUSTRY

IS ONLY ABOUT 50 YEARS OLD SO ANY “TRADITIONAL” ACTIVITY ISN’T SO CARVED IN STONE THAT IT CAN’T BE RESTRUCTURED TO FACTOR

IN NEW PRODUCTS, PEOPLE, SYSTEMS – AND

TESTING PRODUCTS.


LAST WORD

rowing up, many decades ago, we lived next door to an “old guy” who had an immaculate lawn. He worked on it all spring and summer. He was very naturally and expectedly proud of his lawn. He was also

a really nice guy. There was nothing he wouldn’t do for a neighbour in need. Just keep off of his grass. As a young rambunctious lad, I really could have cared less about his lawn. I didn’t get it. It’s just grass! If a stray ball went in to his yard, I would chase after it. Within seconds he’d be out the door screaming at me to get off his lawn. If I happened to let my dog onto his lawn to do his “business” he would practically have a stroke. Looking back, the fertilizer comment I made in response may not have been a great idea.

So here I am some 30 (OK, 40) years later. We are looking at adopting some automated test tools and I have my favourites and my preferences, but so does my significantly younger team mate. But his choice sounds more like a salad ingredient. But, being the open-minded guy that I am, I agreed to give it a try. A half-hearted try maybe, but a try nonetheless. I bought a book on the tool and it seemed pretty straightforward. The examples in the book looked pretty easy to implement. But that’s where the fun ended.

The installation process was a nightmare. First, I had to install something else, and then I had to set environment variables. Then, and only then, could I download and configure it, and make “a salad”. I did all that and it still didn’t work! To say I was steamed would be a huge understatement. I was livid!

That night, as I was lying in bed, too upset to sleep, it hit me. I’d become my neighbour! I didn’t like this young whipper-snapper playing on my lawn. It dawned on me that the best approach was not to force my tool on the team. I’d give the other tool a fair evaluation – and then reject it! But I also thought maybe I should produce a demo of my favourite tool. I’d have my team mate do the same. We’d present both demos to the team and let the team ultimately decide. I agreed, reluctantly, to abide by the team’s decision.

On demo day, I blew the team away. The other demo was kind of lack luster. To be fair, I’ve been doing this a while – I know what I’m doing and I’m very good at it, so I sold my tool. It wasn’t even debated. Probably not very fair either. My approach was universally accepted. I won! I should have been gloating, but why did I feel like garbage?

Did I take advantage of my team mate? Absolutely! I’m older and I’m wiser. I knew how to play the game and I took no prisoners. Afterwards, I congratulated my team mate on a job well done. His demo, while maybe not as polished as mine, was actually very good. I even learned something.

I still felt bad. So I swallowed my pride and asked him to help me install his tool so I could play with it. I still thought my tool was better – for that situation – but now I’ve added a new tool to my tool shed and I’ll use it when the right situation arises. But ultimately, my grass will be greener in the long run.

Don’t get me wrong – I’m hardly a fan. The tool’s installation process is still frustrating, but it isn’t unique to this tool. I recently tried to add something new to my favourite tool and found the experience equally frustrating.

I’m not a programmer. My guess is that the ultimate solution is common sense to a programmer. But not to me. The experience has made me wiser. Now I’ll share my wisdom in the blog-o-sphere and hopefully I can help someone else. That’s just the kind of guy I am.

As for the namesake of the tool – I still despise them. I won’t eat them. If they show up in my salad, I’d give them to my wife or kids. Oh and get off my grass, please!

DAVE WHALENPRESIDENT AND SENIOR SOFTWARE ENTOMOLOGIST

WHALEN TECHNOLOGIESHTTP://SOFTWAREENTOMOLOGIST.WORDPRESS.COM

GET OFF MY GRASS!By Dave Whalen

G

I’D BECOME MY NEIGHBOUR! I DIDN’T LIKE THIS

YOUNG WHIPPER-SNAPPER PLAYING

ON MY LAWN

National Software Testing Conference 2

014

Headline Sponsor

www . s oftware t es t i ngc onference . c om

10% early bird discount if you book before 28th February 2014To book, email: [email protected], quoting: EBNSTC2014

KEYNOTE SPEAKERS INCLUDE:ROD ARMSTRONG Senior director of QA, Expedia

SPEAKERS INCLUDE: • JOANNE HOPKINS

Systems manager, Waitrose Systems

• KAREN THOMAS Senior practice manager, Barclaycard

• PAUL GERRARD TESTA 2013 Lifetime Achievement Award recipient

• CHRIS COMEY Test manager, Testing Solutions Group

• BETHAN LINDLY AND PAUL TWIDELL Microsoft

• CHRIS AMBLER, Director of QA, TCSJohnHuxley

Copyright © 2013 Micro Focus. All Rights Reserved. Portions Copyright © 1994-2009 Borland Software Cooperation (a Micro Focus company).

Silk Performer CloudBurst: limitless numbers,endless possibilities

Like a well-aimed snowball, this time of year is full of surprises. An unexpected rush can crash your site or application when you need it most. So prepare for the unexpected. Use Silk Performer CloudBurst to load and performance test your web and mobile technologies. It works in the cloud across geographies, using limitless user numbers. Get Silk Performer CloudBurst and make snowballs the only ‘meltdown’ you see this year.

Register now for your free 200 Borland Credits at www.borland.com/cloudburst

TEST Magazine - December 2013-January 2014

Documents

issue test automation

test data management

test data preparation

intelligent testing

time testing

technology testing

effective resources