0 © 2014 IBM Corporation Shift Left with Service Virtualization: Rational Test Workbench & Test Virtualization Server Khurram Nizami ([email protected])
0 © 2014 IBM Corporation
Shift Left with Service
Virtualization: Rational Test
Workbench & Test Virtualization
Server
Khurram Nizami ([email protected])
© 2015 IBM Corporation 2
IBM Competitive Project Office
Presentation layer
Integrations, data and business logic
Only one-tenth of an iceberg is above water…
Traditional UI testing:
Automated Integration Testing and Service Virtualization:
The majority of risk in modern systems is seldom tested as it is unseen.
© 2015 IBM Corporation 3
IBM Competitive Project Office
Complexity increases as your points of integration increase….
4 systems 5 systems 12 systems 6 interactions 10 interactions 66 interactions
• Types of systems and
interfaces?
• Cost to interact with
the systems?
• Availability of
systems? Collectively?
• Difficulty in setting up
test systems?
“Lets just deal with it later.….”
© 2015 IBM Corporation 4
IBM Competitive Project Office
Today, integration of enterprise applications still follow the “big-bang” approach …
Mobile
component
Web
component
Mainframe
component
Inte
grat
e
typ. 6 Weeks 1 week
Big Bang!
typ. 6 Weeks 1 week
Big Bang!
Inte
grat
e
© 2015 IBM Corporation 5
IBM Competitive Project Office
Software quality fact: errors caught early cost less to fix . . .
Rela
tive C
ost to
Repair
Acceptance Unit Test Coding Maintenance
1.5x
Stage in which Error is Discovered
“30% of project costs are due to rework
and poor execution of requirements”
“50% of outsourced projects are expected
to under perform”
“34% of software projects are deemed successful
costing $300B annually”
Analysis Design
5x 10x
20x
50x
200x
Bugs/failures are exponentially more expensive the longer they go uncaught
Extends to analysis, requirements, development, and everywhere across the application lifecycle
5
© 2015 IBM Corporation 6
IBM Competitive Project Office
Shift Left – focus on testing high risk areas earlier in the development lifecycle
UNIT SERVICE UI
Automated Integration
Testing
Service Virtualization
Traditional UI Tools
Testing only at the UI level provides little confidence for deployment of complex distributed systems
Testing must be shifted left with focus on testing the highest areas of risk
Service virtualization can be applied at all phases – even allowing integration testing to begin in parallel with Unit Testing.
Shift Left
Development Testing Staging Production
© 2015 IBM Corporation 7
IBM Competitive Project Office
With virtual testing, the “big bang” is eliminated from the process
Requirements
Unit Tests
Performance
UAT
OAT
Big Bang
Integration Tests
System Tests
System
Tests
• Accelerates testing
• Reduces costs
• Lowers risk
7
© 2015 IBM Corporation 8
IBM Competitive Project Office
“We could just write up some stub code…” Not a bad idea, for short lived apps, simple scenarios and limited
sets of integrations…
Hybrid Cloud DevOps
public class TestStockAnalyzer {
public void TestMethod1()
{
// Arrange:
var analyzer = new StockAnalyzer();
// Act:
var result = analyzer.GetContosoPrice();
// Assert:
Assert.AreEqual(123, result); // Why 123?
}
}
public class StockAnalyzer {
public int GetContosoPrice(){
var stockFeed = new StockFeed();
return stockFeed.GetSharePrice("COOO");}}
}
public class StockFeed {
//this stuff isn’t ready, isn’t available yet, is expensive to use, or
some other reason…
}
public class TestStockAnalyzer {
public void TestMethod1()
{
// Arrange:
var analyzer = new StockAnalyzer();
// Act:
var result = analyzer.GetContosoPrice();
// Assert:
Assert.AreEqual(123, result); // Why 123?
}
}
public class StockAnalyzer {
public int GetContosoPrice(){
var stockFeed = new StockFeedStub();
return stockFeed.GetSharePrice("COOO");}}
}
public class StockFeedStub {
public float GetSharePrice(string stock) {
return 123.00;
}
}
© 2015 IBM Corporation 9
IBM Competitive Project Office
Shift Left with Service Virtualization
• Service virtualization
can replace
components and
databases in complex
test scenarios.
• Service virtualization
can work with any
type of test – manual,
automated or
performance
© 2015 IBM Corporation 10
IBM Competitive Project Office
Stubs or “virtual” components allow developers and testers to start integration tests early
Incremental Integration Testing
actual component
a stub that replaces a component that does not exist yet or cannot be accessed in dev
and test
The idea of “stubs” is not new
Developers used them for decades to simulate interfaces and behaviour of
the components that were not available for testing
Components are introduced into the continuous integration cycle in a prioritized,
controlled fashion.
Components not yet built or components that cannot be accessed in the
Development can be simulated and tested against.
© 2015 IBM Corporation 11
IBM Competitive Project Office
IBM Rational Test Virtualization Server
supports multiple platforms and protocols
Transports and Formats The building blocks of an integration test or virtualized service
The transport is the technology that carries your message. E.g. HTTP
The format is the type of message that is carried on the transport e.g. XML
The payload is the contents of the message fields
PAYLOAD
Several transports have
advanced support e.g.
sift and pass through
Formats are presented
in an easily readable
and configurable tree
structure
Fields can be
parameterized using
data sources
© 2015 IBM Corporation 12
IBM Competitive Project Office
Testing and Virtualizing Web Technologies
HTTP(s)
XML REST
SOAP JSON
TCP UDP
XSD
WSDL
WADL
RECORD
Accelerator
Accelerator
• Record transactions
• Or use accelerators such as WSDLs to short cut the process
• Build tests and stubs across a range of supported protocols and formats
© 2015 IBM Corporation 13
IBM Competitive Project Office
IBM Rational Test Virtualization Server
supports multiple platforms and protocols Transport/Protocol Message Formats Technologies
• ActiveMQ • Email (SMTP, IMAP) • Files • FTP/S • HTTP/S • WebSphere MQ • Java RMI • JMS • SAP IDoc, BAPI, RFC • Software AG’s IB & IS • Sonic MQ • TCP/IP • TIBCO Rendezvous,
Smart Sockets & EMS • CICS TG • CICS IPIC • IMS Connect • MQTT
• .Net Objects
• Bytes
• COBOL Copybook
• ebXML
• ASC X12
• UN/EDIFACT
• DFDL
• Fixed Width
• Java Objects
• MIME
• SOAP
• Software AG Broker Docs
• SWIFT
• CHIPS
• FedWire
• TIBCO ActiveEnterprise
• HL7 v3
• XML (DTD, XSD, WSDL)
• ISO8583
• Custom
• CentraSite
• Oracle Fusion
• SCA Domain
• Software AG IS, Broker,
BPMS
• WebSphere Application
Server
• WebSphere Message
Broker
• WebSphere MQ
• Websphere SI Bus
• Sonic ESB
• System z (CICS and IMS)
• TIBCO ActiveMatrix
• UDDI
• Web Services
• WebSphere RR
• WSDL
• BPM
• Databases
• Log Files
Supporting technologies the business depends on Protocols, message formats, and technologies
© 2015 IBM Corporation 14
IBM Competitive Project Office
Integration testing is interrelated and connected to your other aspects of application testing
Integration Testing
Release to Production
Testing
Acceptance
System
Performance
Unit Testing
Analysis
Requirements
Functional Testing
Coding
Main
tenance
All of these tests must be planned. Artifacts created by tests need to be
managed, linked to other development artifacts, and analyzed.
14
© 2015 IBM Corporation 15
IBM Competitive Project Office
Rational also offers a complete suite of testing tools
10
Rational Functional Tester
• Record, enhance, and execute automated tests
• Java and/or Visual Studio generated test scripts
Rational Integration Tester
• Web service visual editors • WS-BPEL testing • Code-free testing
Rational Performance Tester
• Root cause analysis • Performance loads • Automated script capture
and playback
Jazz Team Server
Adapters
Test Planning:
Comprehensive test plan, Shared objectives, Scope,
Timeline, Resources, Risk assessment
Test Execution:
Manual test execution, Use test automation tools,
Record test results, Submit & track defects Reporting & Dashboard:
Status & progress tracking, Customizable live
dashboard, Real-time metrics and reports,
Compliance and quality audit
Test Construction:
Requirement-driven testing, Test environments
coverage, Manual test authoring, Test lab
management
Team Collaboration:
Process enactment and enforcement, reviews, task
management, Rapid team member on-boarding
Rational Quality Manager
Rational Test Workbench
© 2015 IBM Corporation 16
IBM Competitive Project Office
Rational Test Virtualization Server
Webservices Databases MQ Mainframes Middleware and others…
Test Management
Automated Deployment
Rational Quality Manager
Urban Code
Version Control
APM
Automated Testing
Test Data
Continuous Integration
Rational Test Workbench
and others
Jenkins
and others Optim
and others
Service Virtualization
ITCAM
and others
RTC and
others
and Manual Testing
© 2015 IBM Corporation 17
IBM Competitive Project Office
Orchestrate and automate the deployment and testing of applications,
middleware configuration, and database changes into development,
test and production environments.
Databases Internal Messages
Third-party Services
virtual components
Simultaneously
test across
multiple test
stages
Dev QA
IBM Rational Test
Virtualization Server
Combining continuous testing with Continuous Testing from build to production Reliable, repeatable, & traceable deployment automation
IBM UrbanCode Deploy
IBM Rational
Test Workbench
integrated with
Test Environments
Dynamic Infrastructure
Deploy what is ready,
virtualize the rest
Continuously test in
production-like env.
Deploy private, secure
right sized test data
IBM InfoSphere
Optim Test Data Mgmt
UC Deploy Plugins
IBM Rational
Quality Manager
Manage quality across
the delivery lifecycle
© 2015 IBM Corporation 18
IBM Competitive Project Office
Continuous Deployment Continuous Testing
UrbanCode
Deploy
Automated Integration Testing
Service Virtualization
1 2
3
1. UrbanCode Deploy enables continuous deployment of available
environments into the targeted infrastructure
2. RTW/RIT can automatically test the deployment and integration of that
environment - do the components communicate correctly?
3. Service Virtualization allows the testers to:
a. Stand up virtual services in place of components that are
unavailable (not built yet, too expensive, resource contention, 3rd
parties etc.)
b. Plan for test conditions, such as negative testing that may not be
possible with the real components.
= RIT/RTVS
= URBANCODE
Continuous testing and continuous deployment
© 2015 IBM Corporation 19
IBM Competitive Project Office
Continuous testing with virtualized services
Automating deployment and testing to production-like environments
Databases Mainframe applications
Third-party Services
Rational Test
Virtualization Server
Application
changes
being tested
Virtualized Services
Dev QA
Rational Team Concert
Jenkins Build
Deploy
Rational Test Workbench
Test
IBM UrbanCode Deploy
RTW plugin RTVS plugin
© 2015 IBM Corporation 20
IBM Competitive Project Office
RQM and Service Virtualization Invocation
20
RQM RTCP / RTVS RTW / RIT
Associate Test with Virtual Service
Publish Virtual Service
Start Virtual Service
Run Test Stop Virtual Service
Virtual Service Logs linked to Test Case Result
Rational Quality Manager
Rational Test Control Panel
Rational Test Virtualization Server
RTW: Rational Test Workbench
RIT: Rational Integration Tester
Create Test Case Create Virtual Test Service
© 2015 IBM Corporation 21
IBM Competitive Project Office
The Coca-Cola Company decreased production defects by 26-50% with IBM
Business Challenges:
• Quality issues/defects found in production
• Insufficient or unavailable test
environments
The Solution: The Coca-Cola Company
chose IBM’s service virtualization and
automated integration testing solution over
other offerings for the following reasons:
broadest support for the technologies in their
environment; cost of the solution; realizing
increased developer and tester productivity.
Software:
IBM Rational Test Virtualization Server, IBM
Rational Test Workbench
26-50% Decreased
Defects in production
26-50% Accelerated
Project delivery
26-50% Improved
Team productivity
26-50% Reduced
Cost of testing
http://www.techvalidate.com/tvid/1B0-FCA-9AD
© 2015 IBM Corporation 23
IBM Competitive Project Office
Configure Virtualize Model Deploy Execute
Execute
specified
test cases
Listen and
capture
transactions
Build
negative
test case
Refine ROI
Model
behavior of
transactions
Model
negative test
cases
Model
“perfect
storm” test
cases
Apply
metrics to
virtual
models
Deploy
models to
virtual
environment
Configure
SUT
connections
Execute
specified test
cases
Capture test
results
Capture and
measure
results
Recommend
next steps
Install and
configure
SV software
Configure
transactions
/ protocols
Configure
services
Build
additional
AV test
cases
Identify
testing pain
points
Understand
IT
technologies
Review use
cases
Initiate ROI
analysis
Understand
testing
methodology
Discover
Approach to get started with Virtualization