Page 1
Digital Assurance: Develop a Comprehensive Testing Strategy for Digital Transformations
Shamim Ahmed, AVP, Quality Engineering and Assurance
Application Economy: What’s Next
Cognizant
AET02S
Vikul Gupta, Director DevOps, Quality Engineering and Assurance
Page 2
2 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
© 2015 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.
The content provided in this CA World 2015 presentation is intended for informational purposes only and does not form any type
of warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA.
For Informational Purposes Only Terms of this Presentation
Page 3
3 © 2015
Digital Assurance in the context of digital business challenges
Technology Assurance
Agility Assurance
Customer Experience Assurance
Information Assurance
CTO/ CIO
CSO / CDO (data)
CMO /
CDO
CXO / Business
How do I enhance the digital
customer experience?
How do I deliver speed in a
digitally assured business?
How do I secure the hi-performance
digital enterprise?
How do I maximize the
digital value chain?
Assuring Compliance, security and
performance to customers and
organization across geos
Challenges in integrating legacy
systems with digital ecosystems
Assuring “first time right” amidst
increased ecosystem complexity,
shrinking timelines
Test like a customer would – any
device/anywhere/anytime
T h e C h a l l e n g e Fa c e d b y S o l u t i o n
Page 4
4 © 2015
Application level testing Digital product/system/value chain assurance
Guardian for functionality, performance, security Guardian for customer experience & brand
Descriptive metrics Predictive metrics, Quality Intelligence
What’s different in Digital Assurance vs Traditional QA
Tr a d i t i o n a l QA D i g i t a l A s s u r a n c e
Test automationLifecycle automation, early & continuous automation
Robotic process automation
Dev->Test->OpsAgile, Continuous Delivery, DevOps; QA->QE->SET->DET
QA shift left, right, up, down/deep
Page 5
5 © 2015
Digital Technology and
Value Chain AssuranceAgility Assurance Customer Experience Assurance
Security, Privacy, Omni-Channel, UX, Quality Intelligence, Compliance, Reliability Assurance, I18N,Performance
Digital assets, SMAC+
Value Chain TestingDevTOps
Customer Experience
Testing
Information Assurance
Cognizant Digital Assurance Focus Areas
Page 6
6 © 2015
Example: Digital Healthcare Transformation
Limited-Engagement
Highly Engaged, Empowered
Inaccessible
Convenient and 24/7
Transactional, Isolating
Care Team Managed
Office Hours
Virtually Mobile, Anytime
Physician-Centered
Patient-Focused
Isolated Individual
Socially Connected
Sick- care
Health and Well-Being
Limited Consequence
Financial Rewards, Incentives
CHANGING INDUSTRY LANDSPACE
Page 7
7 © 2015
Example: Code Halos in Digital Healthcare
Use
gamification,
apps,and
appropriate
literacy levels
to learn
about
the disease
state
Doctor sends
an email
and also
recommends
an app
Real-time
Patient
coachWearables
help to remind
Inspire, track,
and communicate
Visually
Elegant
Patient
dashboards
show key
measures
that matter
OMG,I HAVE
A SERIOUS
DISEASE
WHAT DO I
DO NOW?
CAN I DO
THIS MYSELF?
HOW DO I
KEEP GOING?
HOW AM
I DOING?
Page 8
8 © 2015
Customer
Service
Individual or Group policy
selectionSales business process
(Prospect and Quote) Member
Enrolment
Claim Management (Claim
Entry and Claim adjudication
and Claim Payment)
Receivables /
Payment
Social Media Big Data Internet of Things Digital Hub Mobile & Smart DevicesCloud Computing
PBM, Payer provider
pharmacy benefits
TPAs
HEALTHCARE VALUE CHAIN
IDENTIFY
SYMPTOMS
CAUSES of
symptoms
PR
OV
IDER
AP
PLIC
AT
ION
CLOUD
MEDICAL
PROCEDURES
PA
YER
AP
PLIC
AT
ION
APPROX.
COST
Suggested
Prescriptions
Liability
Collects
data of
symptoms
PATIENT
SC
EN
AR
IO
Social, Mobile, IoT
& Wearables
Testing, Omni-
Channel testing
CX Testing
Cloud + SOA Testing
(API/Func/Sec/Perf)
Big Data &
Analytics Testing
Omni-channel
API testing
SOA testing
Example: DA Solutions Mapped to Digital HC use cases
Agility Assurance
Page 9
9 © 2015
SMAC Testing
Digital Assurance
Platform
Core Testing
Multi-channel &
IoT Testing
Hybrid Cloud Apps
Testing
Digital Content/Media
Testing, Digital Asset
Management Testing
Deployment Testing
Infra Testing
Business Process
Validation Testing
Customer sentiment Analysis
A/B Testing
DRM Testing
Creation
Aggregation
Distribution
Digital Value Chain Assurance
Page 10
10 © 2015
The impact of Analytics on QA: Digital Assurance Platform
Plan Define Develop Build Test Release OperateProcess
IT Lifecycle Intelligence using Continuous Analytics
Biz
Analytics
Dev
Analytics
Test
Analytics
Ops
Analytics
Cust
Analytics
Analytics
Types
Examples of
Smart QA
based on
Analytics
Feature
ValidationCode-quality
based QA
Schedule &
defect
prediction
A/B Releases,
failure
prediction
Customer
Sentiment
and CX
Page 11
11 © 2015
DevOps
Plan
Define
Develop
Build
Test
Deploy
Release
Operate
Develop
Deploy Operate
Learn
QE QA AA
Requirements
Bugs and feedback
Product
backlog
Ops
backlog
Working
software
Sprint
Monitor
Dev Team OPS Team
QA Shift RightQA Shift Left
QA Sits in the middle
connecting Biz, Dev
and Ops
QA
Sh
ift Up
QA
Sh
ift
Do
wn
Focus on Continuous Quality
Focus on Frictionless Testing
Focus on “Build to Run/Deploy/Perform”, vs “Test…”
QA’s role is changing in a world of DevOps…
Page 12
12 © 2015
DEV-OPS implications for
QA
Quantify impact and test what
matters
Integrate test with Dev. and Ops
• System teams (with Dev and Ops)
• QA tools integrations w/ Dev and Ops
• Code quality-based tests
• CI/CD process integrations
• Automated workflows
• Code change impact testing
• Traceability to environments
• CX focused testing
• Monitoring during tests
• Apps lifecycle analytics & intelligence
Automate aggressively
Virtualize
• Service/App Virtualization
• Network Emulation
• Data Virtualization
• Test Environment
• Device Virtualization
• Test environment
• Test generation & execution
• Test automation
• Test data
• Process and analysis
DevOps Implications for QA
QA Skills Transformation: QA->QE->SET->DET
Velocity
Quality
Page 13
13 © 2015
Testing framework for end to end Digital Stack with IoT
THINGS and PEOPLE
Devices/Sensors
External conditions
External (test) data
Scaling
Simulators and Emulators
Robotic Testing
Non-conventional interactions
Scaling
Omni-channel
Usability
Privacy
NETWORK
Network Virtualization
N/W protocols
N/W conditions
Fault injection
DATACENTER/ CLOUD
APPLICATIONS
Service Virtualization Performance,
Capacity, Scalability, Failover
Security
Omni-Channel delivery
Functionality
Data Integrity
Analytics
CTS
Bluelink
Robowear
Page 14
14 © 2015
Social Driven QA
Earlier if there is a problem with production system, the testing team would be likely to hear through conventional channel such as customer
support, emails or satisfaction survey. Fast-forward to the Digital era
Production Issue User affected Shares experience on Social Media
Before action is taken. Goes viral , reputation damaged
Approach
Social driven testing harvests Social media data , analyze it to exact trends to draw inferences on focus areas to improve product quality/additional validations for QA
Listen Using social media web services (Facebook, Twitter, Foursquare, YouTube, blogs, emails, etc.)
crawls through social media websites
IntegrateHouses social media data and associated BI/analytics structures. Integrates with the
organization’s data warehouse and data mart to bring 360 degree view
AnalyzeMeasures brand’s sentiment on social media sites using Cognizant’s proprietary sentiment
analysis algorithm
TrackMobile Bl ready dashboards and reports that measure brand sentiment analytics, competitive
intelligence & customer insights, social media ROI calculation and marketing campaign
effectiveness
Act Make informed decision, take action and track the results
Page 15
15 © 2015
Case Study: Leading Apparel Manufacturer
Tweets/posts on
products Analyzed
Info extracted in
JSON format using
GNIP
Load Data from
HDFS into Hive for
Querying
Analyze Twitter data
using Tableau reporting
tool
Device
Multiple JSON files stored in HDFS
Trophy
Battery Sync40,000 Tweets on user issues
specific to customer extracted
Validated with test teams,
9 Product Insights/Additional
Test Scenarios on
70 Validations/Scenarios
to improve Product Testing
1.7 MILLION
Page 16
16 © 2015
Key tenets for a successful Customer Experience Testing strategy
Emotion is a key driver Omni Channel Experience
Understanding Customer Intent Multi platform & devicesKey
Tenets
CX is a Continuous Journey Persona Based Test Strategy
Page 17
17 © 2015
The importance and criticality of the Quality Assurance across the User Experience life cycle
UI architecture, content and navigation
Create Story boards & screen flows
Visual design/branding
Design & Build
• Wireframes and mockups to
prototypes through design Iterations
• UI feedback and usability issue list
• High-level UI specifications
RWD design, Scenario and task creation,
consent forms, and focus Group study
Usability, accessibility & compliance
Formative & summative testing
Experience Tracking
Monitoring and analytics
Dashboard and reporting
Design & feature enhancements
Post Production
• Channel integration
• Channel innovation
CET, Usability & comparative
benchmarking
Continuous assessment and testing
Use of production data for further testing
Deploy & Release
Page templates and UI pattern creation
Screen design - modules/use cases
Feedback, messages and error handling
RWD Testing, Multivariate, A/B testing,
Testers as users
UI compliance test against guidelines and
best practices
Issue tracking and fixing
Strategize & Plan
Find customer segments & needs
As-Is assessment
Assess Competitive landscape
Test Plan/Tester Profiles
• Customer insights & user profiles
• Customer Journey maps
• Design blueprint
• KPIs and scorecard
QA involvement in requirement and
customer analysis to build scenarios
User analysis and usability base lining
Use of production scenarios and data
• UI standards, templates and
reusable UI components
• Detailed design specifications
• Visual Design Library
Test Scenario, Test Case, Test Reports, Metrics
CX Testing across the Apps Lifecycle
Page 18
18 © 2015
Application level testing Digital product/system/value chain assurance
Guardian for functionality, performance, security Guardian for customer experience & brand
Descriptive metrics Predictive metrics, Quality Intelligence
Summary: Key Takeways
Tr a d i t i o n a l QA D i g i t a l A s s u r a n c e
Test automationLifecycle automation, early & continuous automation
Robotic process automation
Dev->Test->OpsAgile, Continuous Delivery, DevOps; QA->QE->SET->DET
QA shift left, right, up, down/deep
Page 20
For More Information
To learn more, please visit:
http://cainc.to/Nv2VOe
CA World ’15