Next Gen Software Testing Trends 1
Next Gen Software Testing
Trends
1
Software Testing Trends
2
Software Testing Job Trend - Case for Selenium
3
Selenium & products based on Selenium are the most popular and preferred testing tools
Where is SW Testing industry heading?
What are the Key Influencers
New development methodology trends (Agile)
SaaS / Service based models
Higher emphasis on quality - reputation & business risk
Time to market
Increase in transformational projects
Cost
4
Where is SW Testing industry heading?
Software testing budgets rising and likely to overtake software maintenance budgets
IT budget allocation towards QA projected from 18% in 2012 to 29% in 2017
Increase in transformational projects – mobile, cloud & big data applications
Increase in development means more Risk
Application failure – consumer backlash & reputational risk
Key focus is on cost optimization via process & technology
5
Over 2/3rd of software development projects using agile method to deliver software quickly.
As software releases become more frequent, testing processes have to keep pace
Testing of SaaS Applications
Big push towards Cloud Migration
SaaS has changed the way users think about software
Rather than purchasing, users are now renting
Renting vs Buying mentality - Users will move out quickly if you don’t provide what they want, when they want it
Large SaaS projects are onboarding clients with requirements that are unique & complicated
Verification of data migration (ETL) & functional flow within framework of the offered solution for each client is complicated challenging
Advantages of SaaS - no infra, on demand, cheap, scalable, create and run & access anywhere & anytime
Software Testing becomes extremely important - attention to user experience, user satisfaction, and performance
6
Big Data and Analytics
Rapid growth – 90% of world’s data has been created in the past 2 years (IBM)
With Big data comes bad data
Average organization loses $14.2 million annually because of bad data (Gartner)
75% of businesses waste 14% of revenues due to poor data quality (Experian Data Quality)
Testing Challenges
Testing huge amounts of data and their correctness & accuracy – because the data is
sourced from multiple platforms
Testing focused on data irregularities and quality issues, accuracy of ETL and if source
data is relevant and accurate for analytics
7
Internet of Things (IoT)
IoT covers all devices that connect to Internet
Thermostat, Automobiles, Washing Machines, Smart Watches + 1000’s more that collect, display,
react to and send data all the time
Challenge is for companies that serve data to many devices – they cannot have all of
them for testing
IoT Testing will be very focused – largely dependent on who the customer is, e.g.
Performance & security for internet connected storage devices
Data collection & delivery for medical devices
8
Testing Centers of Excellence (TCOE)
TCOE - A group of people, processes, tools & best practices for enhancing testing
effectiveness
Enterprises tasked with introducing innovative technologies & new systems at record
speed – in-memory databases, mobility, portal, and cloud technologies are driving the
pace of innovation
Many Fortune 1000 firms find TCOE the best approach to reduce IT costs, accelerate
innovation, gain testing competency, and improve the quality of business systems
Push towards Agile & DevOps practices
Continuous & automated testing is becoming a key strategy
9
Testing the SMAC pack
What is SMAC? - Social media, Mobile, Analytics and Cloud technologies
Social media & mobile demands of "zero tolerance" will drive QA demands higher
As more mobile apps come to the market - ones that are built well and offer better value
through frequent feature updates will probably survive - Mobile QA to support all types of
apps becomes very important
SMAC Testing applications need to
Allow insight into customer behavior & become more business centric
Support multiple devices & platforms
Focus on testing algorithms to secure SMAC data and applications
10
Software Testing Trends at Granular
Level
11
Continuous QA (CQA)
What is CQA? Process of executing automated tests as part of the software delivery
pipeline & get immediate feedback on risks associated with software release
Continuous Integration (CI) & Continuous Deployment (CD) aim to introduce smaller code
/ features into production in shorter time,
CQA means test early & often. Integrated approach bridges the gap between CI & CD
CQA needed for faster releases, quality software, catch bugs before deployment
On demand, fast, flexible, comprehensive - traditional QA does not work with CD
Goal – Test software builds as rapidly as they are being produced
12
Crowd-sourcing
Allows for remote usability testing – Testing is user centric
Software is put to test in diverse, realistic environments. It is more reliable, cost effective,
bug free
Pool of testers is diverse and unbiased
Parallel testing by multiple testers makes it faster
13
Cloud based Testing
Ideal for SaaS applications hosted on the cloud
Opportunity for collaborative testing without boundaries
Allows on-demand scalability without infrastructure concerns
Great for small to medium sized development projects
Easy to manage & offer Subscription based model based on usage which is attractive to
software projects of all sizes
Ready testing infrastructure means quick startup times
14
Test Pricing models
Flexible pricing models are becoming more popular
Pay per usage
Use case based
Results based
Time and material
Risk reduction makes it attractive to businesses of all sizes
Free trial periods
Fast evaluation cycles - businesses can’t afford to get caught up in month(s) long proof of
concept (POC)
No long term vendor lock-ins
15
Testing concepts
Testing concepts to reduce risk, cost, time and effort & increase test coverage
Risk driven testing - testing tools that identify what to automate rather than automate everything – e.g. if feature 3 & 5 not tested the risk is 80% compared to 1 & 2 which is 20%
Flexible & configurable codeless testing – Allow business users to actively get involved in testing & automation & perform complex functional tests (record and playback with advanced features)
Cross-browser & multi-device testing – test application compatibility on multiple browsers & user experience in various resolutions on multiple devices
Modular testing – use multiple data sets on base tests. Reuse common data sets in multiple test (ex. User login)
Test management & reporting – ability to effective control a test environment
Quickly identify defect patterns & monitor the pace of corrective measures
Prioritize test execution by criticality or functional areas of the software
Schedule regression / routine tests
Performance Testing – benchmarking & grading application performance over a period of time
16
Security & Penetration testing
Driving forces – rapidly increasing cyber security threats, strict government regulations & compliances, automated testing tools
Growth projected to be from $2.47 billion (2014) to $4.96 billion (2019) – (Markets and Markets)
Pentest is a software attack on a computer system to identify security weaknesses & potentially gaining access to the computer's features and data.
Can help determine whether a system is vulnerable to attack
Goal
Identify vulnerabilities that may be difficult to detect with automated network scanning tools
Assess the potential business & operational impacts
Test ability of network defenders to defend & respond
Justify additional spend on security tools & resources
17
Some helpful tips on selecting a
test automation platform
18
Identify your Pain Points
19
Affordability
Scalability
Complexity
Flexibility
Timeliness
Resources Pain Points
Create an evaluation criteria that covers your
pain points (see example)
Cost
Hardware Cost
Lab Cost
Infrastructure Cost
Resource Cost
Training Cost
Licensing Cost
20
Time
Setup Time
Learning Time
Evaluation Time
Test Creation Time
Ongoing Maintenance Time
Features
Coding effort needed vs. Codeless
Record / Playback
Test Coverage
Test Management
Test Execution
Extensibility
CI/CD tools
Custom scripts
Portability (export/import selenium)
Scheduler
Site monitoring
Responsive testing
Reporting
Data Driven
Multi-Browser testing
API testing
Risk
Startup time
Startup expense
Ongoing expense
Pilot project
Licensing terms
Resource pool
Support
Reliability
Security
Perform Cost/Benefit analysis (see example)
21
Some information on CloudQA
22
CloudQA & how it is adapting new trends
What is CloudQA?
Test automation and test management of web applications
CloudQA design influenced by emerging trends & demands
Make it fast & easy – no programming skills required
Cloud enabled service - zero infrastructure, hassle free startup effort & accessibility
Low risk implementation – time, cost and effectiveness
Appeal to development environments of all sizes & budgets
23
What does CloudQA offer?
Basically, CloudQA is a cloud based test automation tool that has taken care of a lot of work automation testers have to do themselves:
It provides a readymade framework
Has a robust test recorder that even handles recording of complex web technologies and even the ones that the selenium recorder has difficulty with
Has built in test management capability so you can manage 1000’s of test cases
Has a powerful execution engine that runs in Amazon cloud and allows you to run your tests against multiple browsers
Allows massive parallelization that will allow you to run 100’s of “non-sequential” tests in parallel – literally cutting down execution time to minutes instead of hours or days
Has a powerful reporting engine that keeps a track of your run history for months
Integration with Bug reporting, CI/CD tools like Jira, Jenkins, etc and extensions via Zapier
Allows Data driven testing where you can record a test scenario once, create a data template, load new data sets and run them without creating new tests
Allows codeless recording of tests and also allows extensions using javascript for those rare situations where the recorder does not work out of the box
Include API testing
Includes site monitoring
Has a scheduling feature so you can create sets of tests and run them unattended at predetermined times
Benefits of CloudQA over other tools
1. More time savings and staff efficiency
Fastest & easiest setup
Codeless testing, and hassle free maintenance
On-demand or scheduled remote test execution
Quick integration with productivity tools you use (Jenkins, JIRA, Slack, CircleCI, Teamwork + many more)
Comprehensive Notifications, Reports and Dashboards
2. Higher quality in business processes and software applications
More web technology coverage than any other tool (Angular JS, Ember JS, iFrames, Telerik)
Data driven testing to reduce test creation & maintenance time
Modular & Extensible without the complexity of custom coding
Cross platform and browser compatible
3. Minimal Risk
Evaluation Risk - get started in minutes & find out quickly if CloudQA works for you
Resource Risk - Zero Infrastructure. Extremely short learning curve - use existing resources for testing
Cost Risk – Pay as you go subscription model vs. licensed model with vendor lock-ins
25
Please feel free to contact CloudQA if you have any
Test Automation related questions or inquiries.
Thank you!
visit https://cloudqa.io [email protected]
26