IONA Confidential Accelerating Project Delivery QA Frameworks for Distributed Systems
Jan 13, 2016
IONA ConfidentialIONA Confidential
Accelerating Project Delivery
QA Frameworks for
Distributed Systems
80% of Global Telecom 70% of Financial Services in Global 100 Blue Chip System Integrator Partners
EMEA HQ in Dublin, Ireland US HQ in Massachusetts APAC HQ in Tokyo, Japan
Worldwide presenceWorldwide presence
Founded in 1991 Publicly traded since 1997 $50+ million cash on hand No debt
NASDAQ:IONA
Solid business - profitable growthSolid business - profitable growth
Deliver high performance integration software for mission critical applications
Make heterogeneity an asset, not a liability Deliver on the value proposition of
standards
Our Approach: Making Software Work Together™Our Approach: Making Software Work Together™
Customers include world’s largest firmsCustomers include world’s largest firms
… at a glance
Integration Issues Delay ProjectsLots of tools and practices focus on application development and testing, few focus on Integration
We find three major issues: Incomplete integration specs Poor team communication Late integration testing
Define Construct Test Deploy Manage
Governance
Productivity
Technology Implementation, Training & Support
Runtime Components
Development Governance Runtime Governance
Authoring Tools
Professional Services
Development Tools
Testing Tools
Need integration across the software development lifecycle
Challenges of Distributed Development
Isolated teams operating on discrete requirements
Inter-dependencies drive serialized development
Consensus on specifications more difficult to accomplish
Conformance to specifications difficult to monitor
Integration Defects Discovered later
IONA QA Solution FrameworksReduce Definition DefectsIdentify and eliminate IDD content errors and their causes
IDD Audit and Defect Elimination IDD Authoring Solutions Best Practices Training and Consulting
Reduce Communication ErrorsIdentify and eliminate communication errors between groups
IDD Documentation Standards IDD Repository Solutions IDD Change Management Practices
Accelerate Service Testing
Discover system defects earlier and resolve them with less cost Artix Universal Test Harness Artix Certification Kits
Deliver Projects Faster
Reduce Communication Errors
Plan2 Wks
Analysis5 Wks
Build8 Wks
Test6 Wks
Design3 Wks D
ep
loy
Reduce Definition Defects Accelerated Testing
PRE
Analysis5 Wks
Build7 Wks
Test4 Wks
Plan2 Wks
Design2 Wks
De
plo
y
POST21
Weeks
25
Weeks
Solutions
Solution - Reduce Definition Defects
Traditional Interface Specs Text-based and
Difficult to Create- Volumes of Information- Disparate Stakeholders
Inconsistent Results- Ambiguous Content- Information Gaps- Extraneous Information
Traditional Interface Specs Text-based and
Difficult to Create- Volumes of Information- Disparate Stakeholders
Inconsistent Results- Ambiguous Content- Information Gaps- Extraneous Information
Distributed Interface Specs Streamline
- Eliminate Excess- Close Gaps
Decompose- Targeted sub-deliverable
Mechanize- Replace text with more
structured formats
Distributed Interface Specs Streamline
- Eliminate Excess- Close Gaps
Decompose- Targeted sub-deliverable
Mechanize- Replace text with more
structured formats
InterfaceSpec
Pt. 1 Pt. 2 Pt. 3 Pt. 4
Customer Case Study•Experienced SOA Shop•Standardized SDLC•Audit early 2006
- 121 Interface Specs- All had been “signed off”.- 716 Defects / 436 Severe - A Severe Defect in every IDD
Customer Case Study•Experienced SOA Shop•Standardized SDLC•Audit early 2006
- 121 Interface Specs- All had been “signed off”.- 716 Defects / 436 Severe - A Severe Defect in every IDD
Reduce Design Cost and Time Contributors more finely targeted Fewer Re-designs for missed Requirements Contribute deliverable components eliminated
Improve Quality Incremental Approach Less Ambiguity in Specifications Changes more Visible to
relevant Stakeholders
Solution ValueDistributed Interface Specs
InterfaceSpec
Pt. 1 Pt. 2 Pt. 3 Pt. 4
Solutions - Reduce Communication Errors
Project Document Management
Centralized repository for synchronization, versioning
Ensures teams work from the same specs, receive updates
Provides project history & audit trail
Bug Tracking Logs and tracks defects from
design to testing Means of communicating audit
results to development teams Drives project dashboard
Integration Dashboard Ensures success criteria is
understood and monitored Exposes false assumptions,
eliminates surprises All stakeholders can see
status of inter-system testing
Solution - Accelerating TestingTraditional Silo Testing Test Tools a Requirement
Costly Simulators- Solution varies by technology- Limited Vendor Tools- Hand-code one-offs
Expensive People Resources- Simulator Coding and Maintenance- Non-intuitive Mechanical Interfaces
Traditional Silo Testing Test Tools a Requirement
Costly Simulators- Solution varies by technology- Limited Vendor Tools- Hand-code one-offs
Expensive People Resources- Simulator Coding and Maintenance- Non-intuitive Mechanical Interfaces
Universal Test Harness One Test Harness for all Silos
- Enabled via IONA’s unique IP
Flexible Tool Selection- 3rd Party Tool- Generated Simulators
Highly Scalable- Unit Testing thru Load Testing
Universal Test Harness One Test Harness for all Silos
- Enabled via IONA’s unique IP
Flexible Tool Selection- 3rd Party Tool- Generated Simulators
Highly Scalable- Unit Testing thru Load Testing
IONA UTH
Middleware 2Middleware 1
Middleware 3 Middleware 4
TestHarness
1
$$$$
Middleware 1 Middleware 2 Middleware 3 Middleware 4
TestHarness
1
TestHarness
4
TestHarness
3
TestHarness
2
Solution Value - Universal Test Harness
Concentrated Investment of … Software Dollars People
Greater People Portability Less retraining between systems
Increased Productivity Eliminate Hand-coded simulators Greater Access to best in class
tools
Improved Process Visibility Normalizes Testing Activities Consolidated Reporting and
Management Capabilities
Customer Case Study
• Enterprise Services Organization- Approx 70 Testing FTEs- Annual Budget: ~US$9M
• Assessment – Mid ’05- Testing FTEs reduced 70-80%- 3 year Cost-savings: US$10M
- Developer FTEs 15%- Improved Coordination- 3 year Cost-savings: ~US$5M
- Reduce Time to Market- Removed 2-3 Weeks per release- 3yr Revenue Impact: ~US$50M
Customer Case Study
• Enterprise Services Organization- Approx 70 Testing FTEs- Annual Budget: ~US$9M
• Assessment – Mid ’05- Testing FTEs reduced 70-80%- 3 year Cost-savings: US$10M
- Developer FTEs 15%- Improved Coordination- 3 year Cost-savings: ~US$5M
- Reduce Time to Market- Removed 2-3 Weeks per release- 3yr Revenue Impact: ~US$50M
Solution - Accelerating TestingTraditional Integration Testing Parallel development in isolation
Integration defects discovered in Test
- Must “assemble” app to discover flaws
- Occurs late in project cycle
Time Consuming Discovery Process
- Tends to discover defects serially
- Defective Elements must be returned to Development
Traditional Integration Testing Parallel development in isolation
Integration defects discovered in Test
- Must “assemble” app to discover flaws
- Occurs late in project cycle
Time Consuming Discovery Process
- Tends to discover defects serially
- Defective Elements must be returned to Development Artix Certification Kits
Interface Simulators- Generated form Interface Spec- Consumers and Providers
Parallel Test during Development- Centrally Defined Tests Cases- Distributed to Developers
Artix Certification Kits Interface Simulators
- Generated form Interface Spec- Consumers and Providers
Parallel Test during Development- Centrally Defined Tests Cases- Distributed to Developers
SpecDevelop & Unit Test
Develop & Unit TestTest
App
App
SpecDevelop & Unit Test
Develop & Unit TestTest
App
App
Generate
Kits
Solution Value - Certification Kits
Reduce Rework Early validation of IDD’s integrity Defects discovered before code
delivery
Client System Server System
Certification Kit
Server Simulator
Client Simulator
InterfaceDefinition(WSDL)
Test Data
Doco
Reduce Overruns and Delays Integration Problems Identified
and Resolved earlier
Improve Quality Subsequent Tests may focus on
more complex scenarios
SpecDevelop & Unit Test
Develop & Unit TestTest
App
App
Generate
Kits
The Federated Development and
Certification Environment Example
The Federated Development and Certification Environment (FDCE) is the set of processes and supporting infrastructure that is needed to support the conduct of net-centric Capability Provisioning Operations
DeliveryDelivery• Train ResourcesTrain Resources• Build out and Deploy Build out and Deploy
Solution ComponentsSolution Components
DeliveryDelivery• Train ResourcesTrain Resources• Build out and Deploy Build out and Deploy
Solution ComponentsSolution Components
Validate ResultsValidate Results• Confirm Value Confirm Value
EstimatesEstimates• Secure ReferenceSecure Reference
Validate ResultsValidate Results• Confirm Value Confirm Value
EstimatesEstimates• Secure ReferenceSecure Reference
Getting Started…
BlueprintBlueprint• Scope Solutions and Scope Solutions and
Delivery ModelDelivery Model• Draft Economic Draft Economic
Value ModelValue Model
BlueprintBlueprint• Scope Solutions and Scope Solutions and
Delivery ModelDelivery Model• Draft Economic Draft Economic
Value ModelValue Model
Validate ApproachValidate Approach• Work with your people Work with your people
to validate findingsto validate findings• Executive Briefing Executive Briefing
Validate ApproachValidate Approach• Work with your people Work with your people
to validate findingsto validate findings• Executive Briefing Executive Briefing
Getting StartedGetting Started• Identify business and Identify business and
technical drivers for technical drivers for accelerating the accelerating the delivery of IT projectsdelivery of IT projects
Getting StartedGetting Started• Identify business and Identify business and
technical drivers for technical drivers for accelerating the accelerating the delivery of IT projectsdelivery of IT projects
DiscoveryDiscovery• Architecture and Architecture and
business review for business review for selected systems and selected systems and projectsprojects
DiscoveryDiscovery• Architecture and Architecture and
business review for business review for selected systems and selected systems and projectsprojects
Optional StepsOptional Steps