INDIUM SOFTWARE An Independent Software Testing Firm Indium Software Performance & Automation Testing Services “Business Assurance through Quality Assurance”
INDIUM SOFTWAREAn Independent Software Testing Firm
Indium SoftwarePerformance & Automation
Testing Services
“Business Assurance through Quality Assurance”
© 2017 Indium Software
Performance Testing Service Overview
2
Focused Performance Testing practice – over past 10 yearsOver 50 Performance test engagements executed.Expertise in HP Mercury, Microfocus, IBM Rational & Open Source tools.Strong focus on Independent, Objective and unbiased analysis.
Performance Test Centre Services:• Load / Stress / Volume Testing• Fail-Over testing• Scalability Testing• Performance Benchmark/Baseline Testing• Performance Test Tool and Process Consulting
Differentiators – Simple, Flexible and Scalable• Process (with toll gate rigor) focus• Transaction based pricing (per script model)• High ROI on QA Investments• Innovative engagement models• Just in time resourcing model – core and flexible
teams.
Validate Architecture | Validate Response times | SLAs | Validate Scalability
© 2017 Indium Software3
Proficiency on Tools• Commercial - HP Loadrunner, MS VSTS, OATS, MF-Silk
Performer, IBM’s RPT• Open Source – Jmeter, Badboy, OpenSTA, Grinder
Key Deliverables• Overall Performance Roadmap• Test Plan for all major Releases• Assess high risk aspects of design w.r.t Technology• Comprehensive Test Summary with Observations ,
Recommendations and Extrapolations
Focus Areas - • Application Benchmarking & Predictive Usage Pattern• WAN simulation Using Cloud based load testing• Infrastructure Upgrade – Validation• Mobile application based Performance Testing• Maturity based Performance Engineering Models
Value Additions• Load Balancer / Memory Bottleneck• Processor Bottleneck in DB & App. Server • Disc I/O constraints / Network Latency• Server Configuration / Network Bandwidth • DB Deadlock / SQL response time / Indexing issues• JVM / JDBC Connections Constraints• Timeout / TCP Connection stability
Performance Test Approach | Overview
© 2017 Indium Software
SMART Performance Test Strategy
Prioritization Matrix for Business Scenarios
# of Concurrent UsersFrequency of transaction(s) usage Customer Facing vs Financial Impact
Define High level scenarios
Business Processes• Sub-Processes
• Transactions or Process Steps
Define Test Data
Volume of Test Data needed (Static vs Volatile)Mining of historical data for quick test data preparation
Performance Test Environment
VPN connectivity – Configure Static IP address (Offshore) within client firewallApplication access – Test ID creation, user privilegesTools – Depend on the feasibility
Readiness to Start
People – Performance Testers with multiple test tool experienceProcess – Performance Testing process, methodology, scripting, execution, Analysis & ReportingTools – Templates, Jump-Start-Kits
4
SMART stands for Script, Model, Analyse, Report and Tune
© 2017 Indium Software
Performance Test Schema
5
PC 1 Application / Production Server
PC 3 PC 4
Performance Test Lab:
To conduct load test, monitor performance, analyze and report, remotely.
Load Generators: For hosting Controller & Load Generators for load generation via LAN/VPN onto Application Server. These could be at same location or at different geographies.
PC 2
VPN/LAN
Virtual UsersLoad
PCOffshore (Indium)
PC
CUSTOMER Infrastructure
© 2017 Indium Software
Performance Test Metrics
6
• OS Resource Utilizations CPU Usage Memory Usage Disk IO Utilization
• Network Utilization Bandwidth Latency Traces
• Servers Utilization (Web /App) Heap Usage & GC JVM JDBC Threads Application Logs
• SLA compliance• Certify the build for production deployment
• Trend analysis• SLA compliance
Metrics
•Test Tool Results•Monitors Data•Diagnostics tool data•Data from Support Teams
Analysis Report
•Execution & Monitors data analysis•Log analysis
•Diagnostics report•Bottlenecks and Root causes
Execution Report
•Response times•Server resource usage statistics•Defect tracking and comparison
Test Summary Report (TSR)
•Consolidated report of al the tests conducted•Recommendations
© 2017 Indium Software7
Recommendations
Usage distribution – Response time
Server utilization
Performance Test metrics
Deep Dive Performance Dashboard
Artefacts (Samples) – Performance Tests
© 2017 Indium Software
Test Automation Portfolio
8
• Functional Test Automation, Test Data Generation, Testing Tools Administration, Regression automation
• Multi Browser / Multi OS• Multi Device – PC / Mobile / Tablets
• Usage of reusable functions to ease coding • Reusable Automation Frameworks with proven track record in effort reduction • Process , checklists and templates to act as jumpstart kit
• Web Technologies - Java, .NET, LAMP• Enterprise Apps – SAP, Oracle, Peoplesoft, Salesforce.com• BI / DW, Big Data• Mobile, Social Media, Cloud, Analytics
Services
Jump Start
Technologies
© 2017 Indium Software
Tools• Commercial - HP QTP, IBM Rational
Functional Tester, Silk Test, TestComplete, Ranorex
• Open Source - Selenium RC, SOAPUI, Watir
Key Deliverables• Test Strategy / Plan• Test Automation Framework• Automation Suite• Test Automation Execution SummaryFocus Areas – <<Customer Name>> : • Define the process for the automation
suite management and maintenance Lorem Ipsum
• Implement Test Automation Framework and develop Test Automation Suites for Smoke , Sanity & Regression
• Automation suite should ensure that , the application build is stable across all major / minor releases which could include defect fixes or new feature enhancements and extend good test coverage
Value Additions• Reporting alerts over email / SMS• Cloud ready• Interface with multiple tools• Hybrid automation framework
9
Test Automation Approach | Overview
© 2017 Indium Software
Indium’s Selenium Automation Framework Enhanced (iSAFE) Framework
10
Framework Layer
Mobile A
pplication Under Test
Automation Suite
Automation
Scripts
Generic
Libraries
Input Data
Driver Script
Common
Libraries
Object Repository
Config and Global Variables Reporting Engine
Scheduler
Start up/Clean Up
Properties
Execution
© 2017 Indium Software
iSAFE Capabilities
11
iSAFE Capabilities
Test Reporting in multiple formats
with historical graphs
Ability to work with multiple OS /
Browsers / Devices
Ease of Setup and Execution resulting in
minimum learning
Fully Configurable Tests using
Spreadsheets
Support for API and DB Testing
Email/SMS Text
Status/Voice calls to
Stakeholders
Target based & parallel
execution using Selenium
Grid
Exception Handling and Screen Shot
Management