SAP INNOVATION FORUM ISTANBUL
TITLE SAP Quality assurance solution
Speaker’s Name :Abdullah AL SAUDI (SAP), Eyüp BAY (HPE)
Department :Quality assurance solution
DIGITAL ERA
Connected Innovation
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 2Internal
Agenda
Applications challenges
SAP Quality Assurance Solution
SAP security solution
Testing Center of Excellence & next Steps
Q&A
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 3Internal
Custiomers runs major software operationsapplications
ERPwikis
inventory
management
supply
chainbilling
order entry
PoS
mobile apps
websitepayments
CRM
HR
Embedded
software
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 4Internal
Business Depends on IT
Most enterprises run major software operations.
ERPwikis
inventory
management
supply
chainbilling
order entry
PoS
mobile apps
websitepayments
CRM
HR
Embedded
software
Business survival relies on application agility(while reducing cost and risk)
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 5Internal
Velocity and Quality, find the balance.
Need for
Velocity
Demand for
Quality
50% of consumers
will delete a mobile
app if they encounter
a bugAPMdigest, Feb 5, 2014
30x increase in
application releasesEnterprise 20/20 Research, 2013
Access AnywhereComposite Applications
Big data
Proliferation of Tools
Shift “Left”Visibility
Agility
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 6Internal
Add
In order to Deliver our Business Needs
Append
Adapt
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 7Internal
Velocity and Quality, don’t decide.
Need for
Velocity
Demand for
QualityAccess Anywhere
Composite Applications
Big data
Proliferation of Tools
Shift “Left”Visibility
AgilityVelocity & Quality, we need them
BOTH
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 8Internal
Would you ride this …
….If it has never been tested ?
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 9Internal
Cost per Fault 1x 1x 1x 5x 10x 50x
Why Testing is Critical!
TestingUser
AcceptanceTesting
Deploy to
Production
Planning &Requirements
Design Development
Fault Origination
Requirements
Test Planning
Design
Review
Development
Unit Testing
Functional
Testing
System
TestingProduction
20%13%6% 20% 5%36%Fault Discovery
Software Development Lifecycle
10% 40% 50%
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 10Internal
Why Testing is Critical!
Errors get a lot of publicity….
IT Projects delayed going into
production
Proper Quality Assurance
would DISCOVER issues PRIOR
to release….
“employee was fired ….”
CEO resigned
Lost revenue & huge fines
Brand damage
http://www.informationisbeautiful.net/visualizations/worlds-biggest-data-breaches-hacks/
11© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Getting it wrong looks like…
26% Of IT budget on Testing activities
56% of defects introduced at the requirements phase
82% IT Projects delayed going into production
#1 #1 cause of Dev waste is poor defect mngt and rework
100X Cost to repair a defect in production vs. requirements
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 12Internal
Critical Foundations of an Application
Quality
“Does it WORK
as it needs to?”
Security
“Is it SECURE as
it needs to be?”
Performance
“Will it
PERFORM
under load?”
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 13Internal
ASAP Includes tools, templates and accelerators to help customers define a Quality Assurance Strategy designed to effectively manage the
test management process, governance, and testing solutions that will enable effective execution of their quality assurance lifecycle across
each ASAP phase
SAP’s Quality Assurance Solution Portfolio
SAP Solution Manager
Business BlueprintBusiness Process Change Analyzer
(BPCA)
SAP ASAP Methodology
OperateRealizationBusiness Blueprint Final Prep Go Live SupportProject Preparation
SAP Solution Manager Adapter
Test
ManagementFunctional
TestingRefresh non-Production
DataPerformance Testing
Test Result
Analysis
Virtualize Processes &
Services
Confirm
Successful Test
Executions
Application
Security Testing
SAP Quality Center
by HPSAP LoadRunner by HP
SAP Test Data Migration
Server
SAP Service Virtualization by
HP
SAP Test Acceleration
& Optimization
SAP Fortify by HP and
SAP NetWeaver Application Server,
add-on for code vulnerability analysis
Testing Center of Excellence Supported by: SAP Quality Center by HP, premier edition and SAP LoadRunner by HP, performance center edition
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 14Internal
Best Practices for End-to-End Quality Management
Customer
Project
Like :
Suite on HANA
What to Test for
Suite on HANA
implementation
?
Aspects of
Testing to
consider
Test Execution
& Analysis
Take Decision
for GoLive
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 15Internal
Best Practices for End-to-End Quality ManagementHow to get the list of “What to Test” ?
Current Project
Customer
Project
Like :
Suite on HANA
What to
Test ?
Word / Excel Documents
BPM Tools like ARIS
How to keep “What to Test” up to date with future
enhancements or even changes while the project is running ?
Like : additional modules, new applications, new business processes
How to know what to test for a specific
change event ?
Like : Support Packs, Ehp, Patches, Notes, Process Changes, Bug Fixes
No Metadata or
integration with the
project, the
implementation or
application
?
What is needed ?An integrated way to link the project, the implementation, the business
processes and the application with the Test Requirements
Live Link required
Test
Requirements
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 16Internal
Best Practices for End-to-End Quality ManagementAspects of testing to consider
Current Project Test Cases
Customer
Project
Like :
Suite on HANA
What to
Test ?
Aspects of
Testing to
consider
Manual
Test Cases
Automated
Test Cases
Performance
Testing
Security
Testing
Composite
Application
Testing
Test Data
Manual Testing is very resource intensive and expensive
Every test cycle is the same effort
There will always be some level of manual testing
Minimizing “what to test” is essential
Automated Testing to reduce load on business users
Creation of automated test can be very time consuming
Not every test case makes sense to get automated
Typically 50-60% test automation is achievable
Validate response times and system behavior under load
Performance Testing is key for modern application
Customer facing / Mobile Application Performance Testing
Cannot be done in a manual fashion
Today’s application are not standalone
Highly integrated - legacy apps, external services, non-SAP
Testing of end-to-end scenarios results in delay
Requirement to virtual external services to remove delays
Validate Application Security is essential in today’s time
Hackers are attacking external and internal systems daily
Impact can be significant – personal, revenue, fines, brand
Security Testing from code to production is required
Test Data is needed in non-prod systems to run any tests
Refreshing non-production systems is very expensive
Subset of data is required in non-production systems
Automation of refresh with scrambling of sensitive data
Test
Requirements
Live Link required
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 17Internal
Test Results
Report
Analysis
Impact
Best Practices for End-to-End Quality ManagementTest Execution & Analysis
Current ProjectTest
RequirementsTest Cases
Customer
Project
Like :
Suite on HANA
What to
Test ?
Aspects of
Testing to
consider
Manual
Test Cases
Automated
Test Cases
Performance
Testing
Security
Testing
Composite
Application
Testing
Test Data
Test
Execution
Defects
Test Execution & Analysis
FULL TRACEABILITY FROM RESULTS TO TEST CASES, TEST REQUIREMENTS TO THE BUSINESS PROCESSES
Live Link required
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 18Internal
Test Results
Report
Analysis
Impact
Best Practices for End-to-End Quality ManagementQuality Assurance Solutions – visit IZ03 on the show floor
Current ProjectTest
RequirementsTest Cases
Customer
Project
Like :
Suite on HANA
What to Test ?
Aspects of
Testing to
consider
Manual
Test Cases
Automated
Test Cases
Performance
Testing
Security
Testing
Composite
Application
Testing
Test Data
Test
Execution
Defects
Test Execution & Analysis
Solution Manager
Blueprint
BPCA
Test Scope
Optimization
SAP Quality Center by HP
SAP Service
Virtualization
by HP
SAP
LoadRunner
by HP
SAP Quality
Center by HP
Sprinter
SAP Test
Acceleration &
Optimization
SAP Test Data
Migration
Server
SAP Fortify by
HP
SAP CVA
SAP AGS provides free-
of-charge Expert Guided
Implementation (EGI)*
* For Enterprise Support Customers https://support.sap.com/support-programs-services/solution-manager/training-services.html
Live Link required
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 19Internal
Try:
http://www.informationisbeautiful.net/v
isualizations/worlds-biggest-data-
breaches-hacks/
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 20Internal
The Challenge of Security
In order to secure an application, all of its components, functions,
infrastructure and the related threats must be understood
In order to break an application, only one flaw in any of its
components/functions or the infrastructure may be enough
The problem:
• Each new technology brings with it new vulnerabilities
• Firewalls, intrusion detection systems, signatures and
encryption alone cannot make an application secure
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 21Internal
Application Hacks Increasing in Volume and Impact
This is an issue all our customer are facing
Reported Hacks between 2004 – 2010
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 22Internal
Application Hacks Increasing in Volume and Impact
This is an issue all our customer are facing
Reported Hacks between 2010 – 2015
BREACHED
56 million customers
Malware installed on cash
register system across 2.200
stores syphoned credit card
details of up to 56 million
customers May be the same
group Of Russian and Ukrainian
hackers responsible for the
data breaches at Tarqet, Sally
Beauty and P.F. Chang s,
among
Others. 80 million customers
Second largest health insurer in the US
Feb 2015 Names, dates of birth.
member ID/ social security numbers.
addresses. phone numbers,
email addresses and
employment information.
76 million customers
July 2014 The US's largest
bank was compromised by
hackers,
stealing names, addresses.
phone numbers and emails of
account holders. The hack
began in June but was not
discovered until July, when the
hackers had already obtained
the highest level of
administrative
privilege to dozens of the bank's
computer servers.
145 million customers
The company has said hackers
attacked between late February
and early March With login
credentials obtained from
small number" of employees
They then accessed a database
containing all user records
and coped "a large part"
of those credentials.
Occurred Sep 2014.
Revealed
Feb 2015 Names &
license
plates of 50,000 driver
partners.
Third big data breach from
Citigroup: The personal
information
150,000 consumers who went
into bankruptcy between 2007
and 2011 — including their
social security numbers -
were exposed after Citi failed
to properly redact court records
before they were put on the
Public Access to Court Electronic
Records (PACER) systems.
Frequent flyer
accounts
-tens of
thousands.
Malware was
discovered in
the credit & debit
card processing
systems at 51
branches in
24 states.
2 million customers
An IT contractor for the
firm used his deep
access to the telecom
giant's system to copy
customer names and
bank account details.
1.16 million customers
Staples says 1.16 million
credit card numbers stolen
in breach. malware
infected the checkout
stations at 115 of its 1,400
U.S. stores
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 24Internal
The Incident
• PlayStation Network breach reported April 2011
• 77M customer accounts compromised
• PS Network completely offline for 25 days
• Total cost of damages / loss > $171M
What’s the Worst that Could Happen?
The Attack
• DDoS attack followed by SQL Injection
• 130+ servers completely compromised
• Account data, credit cards, email addresses stolen
• Required full network shutdown to contain
• More than just PlayStation Network…
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 25Internal
Heartland cybercrime case
1. 2008: Albert Gonzalez and 2 Russian co-conspirators gained access to Heartland systems through a personnel application (SQL Injection)
2. Attackers injected code into data processing network and installed a sniffer malware that was able to see credit card numbers and other details.
3. After being alerted by Visa and MasterCard of suspicious card transactions activity Heartland called U.S. Secret Service and hired two breach forensics teams to investigate
4. Jan 20, 2009: Breach reported by Heartland
• At least 650 financial institutions affected
• 94M credit records stolen
• Fines levied to banks > $6M
• Total cost of damages / loss > $140M
5. At the time, the Heartland breach was the largest identity theft case ever
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 26Internal
of breaches
are reported
by a 3rd party%
Understand the risk
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 27Internal
NetworksHardware
Security Measures
• Switch/Router security
• Firewalls
• NIPS/NIDS
• VPN
• Net-Forensics
• Anti-Virus/Anti-Spam
• DLP
• Host FW
• Host IPS/IDS
• Vuln. Assessment tools
Security Targets are Evolving
Cyber Attacks Are Targeting Application Layer Vulnerabilities
Intellectual
Property
Customer
Data
Business
Processes
Trade
Secrets
Applications
84%
of breaches occur
at the application layer
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 28Internal
Security considerations for Internal only applicationsExamples of attacks for internal only applications
Your Digital Enterprise
App
Are those users secure ? Are those applications secure ? Is the data secure ?
Attacks going to
employee’s for
example via a
malicious email
aka :Trojan horses, Login
Spoofing, Virus, Worms ,
DoS, Man-in-the-middle*
Logic Bombs
Trap Doors*
……
Temporary workers to seasonally expand
workforce – potentially limited security
validation
Negligent/unintentional or unknowingly
employee executes steps they are not
supposed to do
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 29Internal
Ensure Application Security with end-to-end Solution
Find vulnerabilities in the
running application
Manual Application
Penetration Testing
Automated Application
Vulnerability Scanning
DAST
Dynamic Application Security
Testing
SAP NetWeaver Application Server,
add-on for code vulnerability analysis (CVA)
Manual Source
Code Review
SAST
Static Application Security
TestingFind vulnerabilities analyzing
the sources
Automated Source
Code Analysis
SAP Fortify by HP &
Finding security issues at design time instead of in production is easier and less expensive!
Management Platform for Monitoring, Auditing, Analysis, Reporting
ABAP
non-ABAP
non-SAP
with SAP Fortify by HP and SAP CVA (code vulnerability analysis)
Demos
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 30Internal
SAP Fortify by HP - Componentsincluding SAP NetWeaver Application Server, add-on for code vulnerability analysis (CVA)
Audit Workbench
Demos
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 31Internal
Fortify Strategy
Assess
Find security vulnerabilities in
any type of software
SAP, Mobile, Web, Infrastructure
Assure
Fix security flaws in source
code before it ships
Secure SDLC
Protect
Fortify applications against
attack in production
Logging, Threat Protection
Software Security
Assurance (SSA)
In-house Outsourced Commercial Open source
Application
Assessment
Application
Protection
1 2 3
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 32Internal
Dynamic Analysis – What would a hacker do?
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 33Internal
No documented QA processes
Level 1
Project Testing
Project
Focus
Quality Management Maturity Roadmap
Project based People, Processes & Tech.
Majority Manual Testing
Level 0
Manual Testing
Level 2
Product Utility
Centralized & Standardized Testing Administration
QA processes but for individual projects
Security, Policy & Compliance Testing
Centralized & Standardized Tech.
Best Practices Adoption
Service Bureau
Integrated Testing & Remediation for Security/Compliance
Level 3
Service Utility
Process
Standards
Centralized People, Process & Tech.
Process Governancefor Testing & Quality
Thought-leadership for Enterprise Influence
Full-lifecycle approach for security & integration between apps & Ops
Level 4
Center of
Excellence
Center of ExcellenceReactive to Predictable
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 34Internal
Enterprise Testing Center of Excellence
PMO
Business
Development
Project
Management
Infrastructure
Security
Quality
Framework
Infrastructure
Support
Training &
Support
Domain &
Technology
Expertise
Test Center of Excellence
Strategy &
Methodology
Governance
Test Infrastructure
Metrics & SLA
Standardization &
Optimisation
Tools & Techniques
Resourcing
(Ramp up / Ramp Down)
Continous
Improvement
Client GroupsSupport
Groups
Automation &
Innovation
Delivery
Excellence
Board
GovernanceProven
Process
Reusable
Assets
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 35Internal
Test Factory
Bundle of 50 test cases
Additional Execution Cycle
Bundle of 100 test cases
Input OutputProcess
Automated Test Factory
Performance Test Cycle & Tuning
© 2016 SAP SE or an SAP affiliate company. All rights reserved. 36Internal
SAP QAS Simplifies & Speeds Business Innovation
SAP Value Proposition = Competitive advantage through technology innovation
Achieve Quality at Velocity to gain benefit requires new technology = SAP Quality Assurance
Benefits of testing for SAP: Ensuring that SAP
applications delivers the expected benefits and
return in a fast and easy was as possible
Risk of NOT testing: Software failure in production
– leading to reduced productivity, lengthy repairs,
lost data and potential for millions in lost revenue
and fines
‘Must have’ to achieve benefit = Functional, Performant & Secure Applications
FE
AT
UR
ES
VA
LU
E P
RO
PO
SIT
ION
Functional Testing“Does it WORK as it needs to?”
SAP Quality Center by HP
SAP Unified Functional Testing by HP
Performance Testing“Will it PERFORM under load?”
SAP Performance Center by HP
SAP Service Virtualisation by HP
App Security testing“Is it SECURE as it needs to?”
SAP Code Vulnerability Analysis
SAP Fortify by HP
# 1 # 1 # 1
© 2016 SAP SE or an SAP affiliate company. All rights reserved.
Thank you
Contact information:
Abdullah AL SAUDI
Senior Engagement Manager – QAS MiddleEast, North Africa & Turkey
SAP UAE-Dubai
M +971-564164260