TL PM Tutorial
10/1/2013 1:00:00 PM
"Security Testing for Testing
Professionals"
Presented by:
Jeff Payne
Coveros, Inc.
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
Jeff Payne
Coveros, Inc.
Jeff Payne is CEO and founder of Coveros, Inc., a software company that builds secure
software applications using agile methods. Since its inception in 2008, Coveros has become a
market leader in secure agile principles and has been recognized by Inc. magazine as one of
the fastest growing private US companies. Prior to founding Coveros, Jeff was chairman of the
board, CEO, and cofounder of Cigital, Inc., a market leader in software security consulting.
8/20/2013
1
1 © Copyright 2011 Coveros, Inc.. All rights reserved.
Security Testing for Test
Professionals
2 © Copyright 2011 Coveros, Inc.. All rights reserved.
Trainer
Jeffery Payne
Jeffery Payne is CEO and founder of Coveros, Inc., a software company that helps organizations accelerate the delivery of secure, reliable software. Coveros uses agile development methods and a proven software assurance framework to build security and quality into software from the ground up. Prior to founding Coveros, Jeffery was Chairman of the Board, CEO, and co-founder of Cigital, Inc. Under his direction, Cigital became a leader in software security and software quality solutions, helping clients mitigate the risk of software failure. Jeffery is a recognized software expert and popular speaker at both business and technology conferences on a variety of software quality, security, and agile development topics. He has also testified before Congress on issues of national importance, including intellectual property rights, cyber-terrorism, Software research funding, and software quality.
8/20/2013
2
3 © Copyright 2011 Coveros, Inc.. All rights reserved.
Coveros helps organizations accelerate the delivery of secure, reliable software
Our consulting services: – Agile software development
– Application security
– Software quality assurance
– Software process improvement
Our key markets: – Financial services
– Healthcare
– Defense
– Critical Infrastructure
About Coveros
Corporate Partners
4 © Copyright 2011 Coveros, Inc.. All rights reserved.
Agenda
Introduction to Security Testing
Security Testing Framework – Steps in security testing – Security test planning – Security test tools
Wrap up
8/20/2013
3
5 © Copyright 2011 Coveros, Inc.. All rights reserved.
Expectations
What are your expectations for this tutorial?
What do you wish to learn?
What questions do you want answered?
6 © Copyright 2011 Coveros, Inc.. All rights reserved.
Introduction to Security Testing
8/20/2013
4
7 © Copyright 2011 Coveros, Inc.. All rights reserved.
When you hear the term “Information Security” and
“Security Testing”:
What do you think they mean?
What comes to mind?
What is Information Security?
8 © Copyright 2011 Coveros, Inc.. All rights reserved.
Definition of Information Security
Information Security means protecting information and information systems from unauthorized access, use, disclosure, disruption, modification, perusal, inspection, recording or destruction.
The key concepts of Information Security include: – Confidentiality – prevent the disclosure of information to
unauthorized individuals or systems
– Integrity – data cannot be modified undetectably
– Availability – data and systems are available in an uninterrupted manner
– Authenticity – ensure that data, transactions, communications or documents (electronic or physical) are genuine
– Non-Repudiation – ensure that someone cannot deny something
What is Information Security?
8/20/2013
5
9 © Copyright 2011 Coveros, Inc.. All rights reserved.
The Software Security Problem
Our IT systems are not castles any longer!
10 © Copyright 2011 Coveros, Inc.. All rights reserved.
Why Software Security is Important
8/20/2013
6
11 © Copyright 2011 Coveros, Inc.. All rights reserved.
How to Define Security Risk in Software
Common Security Nomenclature – Risk: a possible future event which, if it occurs, will lead to an
undesirable outcome
– Threat: A potential cause of an undesirable outcome
– Vulnerability: Any weakness, administrative process, or act of physical exposure that makes an information asset susceptible to exploit by a threat.
– An exploit is a piece of software, a chunk of data, or sequence of commands that takes advantage of a vulnerability in order to cause unintended or unanticipated behavior to occur on computer software, hardware, or something electronic.
– Attack: the approach taken by a threat to exploit a vulnerability Denial of service, spoofing, tampering, escalation of privilege
Understanding Risk
12 © Copyright 2011 Coveros, Inc.. All rights reserved.
What? How?
Security Testing is testing used to determine whether an information system protects its data from its threats.
Security Testing is not a silver bullet for your enterprise security. Security Testing doesn’t fix your security, it only makes you aware of it. Security must be built into your software
A sound Security Testing process performs testing activities:
– Before development begins
– During requirements definition and software design
– During implementation
– During deployment
– During maintenance and operations
Security Testing
8/20/2013
7
13 © Copyright 2011 Coveros, Inc.. All rights reserved.
Security Testing Case Study
Your company, SecureTelco, has developed an instant messaging program to be used for private use in customers homes and for companies and government agencies.
SecureChat requires users to sign up with an account prior to using the system. After authenticating with a username and password, each user can message other users and expect their conversations to be private.
Users have the ability to add/remove friends from their contact list, search for friends based on their email, block users from IMing them, become “invisible” to all users on demand.
Messages archives and activities logs document user behavior and can be retrieved by the user or a SecureTelco Administrator through the application or by the administrative console, respectively.
Exercise
14 © Copyright 2011 Coveros, Inc.. All rights reserved.
Security Testing Framework
8/20/2013
8
15 © Copyright 2011 Coveros, Inc.. All rights reserved.
“Testing” before development begins is really a QA function to assess the readiness of the organization to build secure software applications.
Always remember that security testing evaluates the security posture of your applications, it does not build security in.
Irrespective of your findings, do not become the “quality police”.
Overview
Security testing before development begins
16 © Copyright 2011 Coveros, Inc.. All rights reserved.
Understand the policies and standards that have been adopted by the organization and their relationship to software security
Examples: – Privacy policies regarding your customer data
– Service level agreements with clients
– IT security standards you must adhere to
– PCI compliance activities for credit card transactions
Your goal is to understand these policies and standards to the level that will allow you to validate security requirements and effectively test the end product against them
Review Security Policies and Standards
Security testing before development begins
8/20/2013
9
17 © Copyright 2011 Coveros, Inc.. All rights reserved.
If the security of your software is an enterprise concern, the development team should be adhering to a defined secure software development lifecycle model.
– Defines development activities that builds security in
– Defines security testing activities performed by appropriate parties (development, testing, security org, operations, etc.)
Common secure software development models – Microsoft’s Secure Development Lifecycle (SDL)
– Coveros SecureAgile process
– There are others as well
Secure software standards – Secure coding standard
Review Secure Software Development Lifecycle
Security testing before development begins
18 © Copyright 2011 Coveros, Inc.. All rights reserved.
Testing activities during requirements definition and software design focus on assuring that security has been effectively integrated into software requirements and the overall architecture and design of the product
Typical activities include: – Security requirements development/validation
– Architecture and design reviews
– Threat modeling
– Test strategy and planning
Overview
Security testing during definition and design
8/20/2013
10
19 © Copyright 2011 Coveros, Inc.. All rights reserved.
Software Requirements
Functional Requirements: These are statements of services the system should provide, how the system should react to particular inputs and how the system should behave in particular situations.
What each feature within the software should do
Non-Functional Requirements: These statements describe additional requirements that are not associated with individual functional behaviors. These statements include information about: reliability, configurability, availability, performance, etc.
What quality goals must the entire software system achieve
Security testing during definition and design
20 © Copyright 2011 Coveros, Inc.. All rights reserved.
Security testing during definition and design
Security Requirements
Security Requirements describe functional and non-functional requirements that need to be satisfied in order to achieve the security attributes of an IT system or application.
What does that mean?
Functional Security Requirements
Additions to functional requirements that define what the software should not do.
Non-Functional Security Requirements
Additional non-functional requirements that define what overall security the system must provide
8/20/2013
11
21 © Copyright 2011 Coveros, Inc.. All rights reserved.
Security testing during definition and design
Example Security Requirement
Functional requirement:
SecureChat login screen shall accept a valid username/password pair and allow system access
Functional requirement that includes security:
SecureChat login screen shall accept valid username/password pairs and allow system access.
• Entering either an invalid username or invalid password will result in the display of the message “Invalid username or password” on a redisplay of the login screen after both a username and password are entered
• Three successive invalid login attempts from a particular machine will lock the user’s account and display the message “User Account Locked, Call System Administrator” on a redisplay of the login screen. Subsequent valid login/password pairs will not allow system access until the account is unlocked by the system administrator
22 © Copyright 2011 Coveros, Inc.. All rights reserved.
Security testing during definition and design
Example Security Requirement
Functional requirements:
SecureChat user shall choose a userid and a password for their account during registration
Functional security requirement:
SecureChat user shall choose a userid and a password for their account during registration
• Userid shall be unique within the system
• Userid shall consist of alphanumeric characters
• Password shall be at least 12 characters long and include at least one capital letter, one special character, and one whole number
8/20/2013
12
23 © Copyright 2011 Coveros, Inc.. All rights reserved.
Examples of Non-Functional Security Requirements
SecureChat shall ensure that data is protected from unauthorized access at all times.
SecureChat shall have an availability of 99.9%.
SecureChat shall process a minimum of 8 transactions per second.
Each SecureChat build shall undergo secure code review prior to release.
All communications between the SecureChat client application and the SecureChat central servers shall be encrypted.
Security testing during definition and design
24 © Copyright 2011 Coveros, Inc.. All rights reserved.
Architectural and design reviews focus on determining whether the stated architecture / design enforces the appropriate level of security as defined in the requirements.
Typically performed by security architects and/or other software leads within the organization.
Examines these artifacts for flaws such as: – Violation of trust boundaries
– Distributed control of authorization
– Custom algorithms for cryptography / random number generation
Architectural and Design Reviews
Security testing during definition and design
8/20/2013
13
25 © Copyright 2011 Coveros, Inc.. All rights reserved.
Design Flaws vs. Implementation Bugs
Flaws (Design Defects) – Misuse of cryptography
– Compartmentalization problems in design
– Privileged block protection failure
– Type safety confusion error
– Insecure auditing
– Broken or illogical access control
– Method over-riding problems
Bugs (Implementation Defects) – Buffer overflows
– Cross site scripting
– Race conditions
– SQL Injection
26 © Copyright 2011 Coveros, Inc.. All rights reserved.
Threat modeling – a process by which any risks to a piece of software are identified and mitigated
A variety of approaches exist for doing threat modeling
Microsoft STRIDE model – Diagram your system – high level dataflow diagrams
– Identify threats (risks) – each type of entity/interaction has enemies
– Mitigate threats (risks) – determine security controls
– Validate mitigations – test effectiveness of these controls
Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, Escalation of Privilege (STRIDE)
Threat modeling for risk assessment
Security testing during definition and design
8/20/2013
14
27 © Copyright 2011 Coveros, Inc.. All rights reserved.
Fixing the Problem – One DoD Initiative
0.00
20.00
40.00
60.00
App1 App2 App3 App4 App5 App6
Critical/High Vulnerabilities Per 1,000 Lines of Code
Initial Follow-On
But there are 1,000’s of apps … do the math
28 © Copyright 2011 Coveros, Inc.. All rights reserved.
Identifying threats and flaws in your design only result in better security if the flaws are mitigated to minimize the threat.
But at what cost to the organization?
What benefit?
How do you convince management to fund mitigation efforts?
Assessing your risk – Answers the ‘so what?’ question
Security testing during definition and design
8/20/2013
15
29 © Copyright 2011 Coveros, Inc.. All rights reserved.
Information on design flaws/vulnerabilities and known threats from our threat model are often combined together to estimate the likelihood and consequence of a flaw/defect resulting in significant business impact
Risk Assessments
Security testing during definition and design
Not a PriorityNot a PriorityNot a PriorityMinor or cosmetic
Not a PriorityPriorityHigh priorityBusiness concern
PriorityPriorityHigh priorityBusiness-critical
UnlikelyLikelyHighly Likely
Not a PriorityNot a PriorityNot a PriorityMinor or cosmetic
Not a PriorityPriorityHigh priorityBusiness concern
PriorityPriorityHigh priorityBusiness-critical
UnlikelyLikelyHighly Likely
30 © Copyright 2011 Coveros, Inc.. All rights reserved.
Risks are placed in appropriate categories based upon understood consequence and likelihood of occurrence
– Consequence – depends upon your business and market
– Likelihood – depends upon your risks and threats
Risk Assessments Results
Security testing during definition and design
Not a PriorityNot a PriorityNot a PriorityMinor or cosmetic
Not a PriorityPriorityHigh priorityBusiness concern
PriorityPriorityHigh priorityBusiness-critical
UnlikelyLikelyHighly Likely
Not a PriorityNot a PriorityNot a PriorityMinor or cosmetic
Not a PriorityPriorityHigh priorityBusiness concern
PriorityPriorityHigh priorityBusiness-critical
UnlikelyLikelyHighly Likely
Denial of
service Stealing of
secrets
Inappropriate
access
Tampering
8/20/2013
16
31 © Copyright 2011 Coveros, Inc.. All rights reserved.
Functional Security Requirement SecureChat Authentication Requirements
– When a user attempts to authenticate with a valid username and an invalid password, the application shall not authenticate the user and return them to the authentication page.
– The system must alert the user that their attempt to authenticate has failed due to an incorrect password (“Invalid Password”) utilizing the standard error text formatting.
– When a user attempts to authenticate with a invalid username, the application shall not authenticate the user and return them to the authentication page.
– The system must alert the user that their attempt to authenticate has failed due to an incorrect username (“Invalid Username”) utilizing the standard error text formatting.
– What a user attempts to authenticate using a username and a valid password, the application shall authenticate the user and redirect them to the homepage.
What risks/attacks might be possible?
Exercise
32 © Copyright 2011 Coveros, Inc.. All rights reserved.
Security test strategy
Security is one aspect of testing that must be incorporated into your test strategy and is typically included in a master test plan
Typical master test plan format – Overview of system
– High level risks and threats to quality
– Types of testing that will help mitigate risks and threats
– Roles and responsibilities
– Automation approach
– Test infrastructure and schedule
Test strategy and planning
8/20/2013
17
33 © Copyright 2011 Coveros, Inc.. All rights reserved.
What should be included?
Describe and detail your process and procedures for security testing
– When should testing begin?
– How are test results reported?
– Who validates and verifies findings/results?
– When are vulnerabilities addressed?
Types of tests you should include in your test plan: – Security Feature Testing
– Risk Based Testing of functional and non-functional requirements
– Internal Penetration Tests
– External (Independent) Penetration Tests
Security test plans are usually separate test plans (for compliance / audit reasons)
Developing a security test plan
34 © Copyright 2011 Coveros, Inc.. All rights reserved.
Integrating security requirements in test plans
Know your Security Requirements – Requirements analysis
It is important that each tester understand the security requirements for your application and what they imply.
Often Security requirements may come in conflict with another type of requirement. If there are conflicts, it is important that you identify those concerns and the requirements are clarified by a Business Analyst.
In most organizations, security requirements are not well defined if it all.
A general rule of thumb: Make sure your core information security concepts are all covered. If not, request that they are.
Understand which security requirements are functional and which are non-functional, this will have an impact how you plan to test them.
8/20/2013
18
35 © Copyright 2011 Coveros, Inc.. All rights reserved.
Integrating security requirements in test plans
Testing Security Requirements
Feature testing covers positive security requirements. This typically ensures the software behaves according to customer expectations.
Example – If security requirements state that the length of any user input must be validated, then a feature test suite should be created to exercise the application inputs and verify that this requirement is implemented correctly.
Testers should also cover negative security testing or Risk-Driven testing. Each test is intended to probe for a specific risk or vulnerability. These risk may have been identified during your risk assessment.
Example – Cross Site Scripting and SQL Injection; These vulnerabilities are not obviously features of the application, therefore the fall under the negative security requirements umbrella.
Security testing tools provide out of the box testing for common web security issues
36 © Copyright 2011 Coveros, Inc.. All rights reserved.
Testing activities during implementation focus on assuring that the software is implemented properly according to its requirements and design
Key activities during implementation include: – Secure code review – identifying security vulnerabilities in source
code
– Testing individual components/features for security
– Testing requirements at the appropriate level
Overview
Security testing during implementation
8/20/2013
19
37 © Copyright 2011 Coveros, Inc.. All rights reserved.
Secure code review identifies vulnerabilities within source code that potentially impact system security.
Examples – Buffer overflows
– Race conditions
Secure code review is a combination of manual and automated analysis
Secure code review is typically done by developers or a dedicated security team
Secure code review
Security testing during implementation
38 © Copyright 2011 Coveros, Inc.. All rights reserved.
The testing of components and individual features will identify code that improperly implements functionality against its requirements.
While some feature testing has historically been done at the system level, more and more of this type of testing today is done on individual units / stories by either a developer or code savvy test engineer.
Review of tests performed at this level should look for common gaps that lead to security issues:
– Inadequate testing of error handling routines
– Insufficient protection during system reboot
– Forgetting to test administrative capabilities
Testing components and features
Security testing during implementation
8/20/2013
20
39 © Copyright 2011 Coveros, Inc.. All rights reserved.
Security testing during implementation
Testing common security controls
Due to the security-critical nature of many of our applications, it is common to see the following security controls implemented within our software.
Each must be validated in order to work!
Authentication & Access Control
Input Validation & Encoding
Encryption
User and Session Management
Error and Exception Handling
Auditing and Logging
Test catalog’s can assure security controls are tested adequately.
40 © Copyright 2011 Coveros, Inc.. All rights reserved.
All About Authentication
When we refer to authentication in computer security, we refer to the process of attempting to verify the digital identity of the sender of a communication.
– A common example of such a process is the login process.
– Authentication always depends upon using one or more authentication category: something I know, I have, I am
Two-factor authentication: factors from two categories – Multi-factor authentication: more than one authentication factor but
can be from the same category
Testing authentication schemas means understanding how the process works and using that information to circumvent the authentication mechanism.
Common Approaches to Authentication
8/20/2013
21
41 © Copyright 2011 Coveros, Inc.. All rights reserved.
Authentication Test Catalog
Credentials transport over an encrypted channel – The tester must try to understand if the data inputted by the user is
transmitted using secure protocols that protect them from an attacker or not.
Testing for user enumeration – The tester must verify if it is impossible to collect a set of valid users
by interacting with the authentication mechanism of the application. This will become useful for brute force testing.
Testing for guessable (dictionary) user accounts – The tester must validate that there are no default user accounts or
guessable username/password combinations
Brute force testing – When dictionary attacks don’t succeed, the tester can attempt brute
force methods to gain access. This is not often easy to accomplish because of time constraints.
Common Approaches to Authentication
42 © Copyright 2011 Coveros, Inc.. All rights reserved.
Authentication Test Catalog (cont.)
Testing for bypassing authentication schema – The tester must validate that other application resources are
adequately protected, and can’t be used to bypass authentication using those other resources.
Testing for vulnerable remember password and password reset features
– The tester must analyze how the application manages the process of “password resets”. The tester must check whether the application allows the user to store passwords in the browser.
Testing for logout and browser cache management – The tester must check that the logout and caching functions are
properly implemented.
Common Approaches to Authentication
8/20/2013
22
43 © Copyright 2011 Coveros, Inc.. All rights reserved.
Authentication Test Catalog (cont.)
Testing for CAPTCHA – Used by many applications to ensure the response is not generated
by a computer, CAPTCHA (“Completely Automated Public Trust test to tell Computers and Humans Apart”) implementations are often vulnerable to various kinds of attacks.
Testing multiple factor authentication – The tester must test the following scenarios:
One Time Password Generator Tokens
Crypto devices like USB tokens or smart cards
X.509 Certificates
Random OTP sent via SMS
Testing for race conditions – The tester must ensure that an unexpected result on a multithread
application doesn’t create an authentication flaw. By their nature, Race Conditions are difficult to test for
Common Approaches to Authentication
44 © Copyright 2011 Coveros, Inc.. All rights reserved.
Authentication Test Catalog (cont.)
Testing for session management schema – The tester must test the security of a session tokens issues to the
client browser: How to reverse engineer a cookie
How to manipulate cookies to hijack a session
Testing for cookie attributes – The tester must check if an application can take the necessary
precautions when assigning cookies and test the cookie attributes.
Testing for session fixation – The tester must validate that an application renews the cookie after
a successful user authentication, so that an attacker could not utilize a session fixation vulnerability.
Common Approaches to Authentication
8/20/2013
23
45 © Copyright 2011 Coveros, Inc.. All rights reserved.
Authentication Test Catalog (cont.)
Testing for exposed session variables – The tester must validate that it is not possible to create a replay
session attack utilizing exposed session information.
Testing for CSRF (Cross Site Request Forgery) – The tester must ensure that there is not a way to force an
unknowing user to execute unwanted actions on a web application they are authenticated on.
Common Approaches to Authentication
46 © Copyright 2011 Coveros, Inc.. All rights reserved.
Password Crackers/Brute Force Tools
Where to use? – When you want to break the default credentials or test your
authentication mechanisms against common security tools.
Free Tools – THC Hydra
– Cain and Abel
– Wfuzz
Paid Tools – John the Ripper
Tools to Support Authentication Testing
8/20/2013
24
47 © Copyright 2011 Coveros, Inc.. All rights reserved.
Risk-based Testing focuses on testing that the risks identified during threat modeling, design reviews, code reviews were properly mitigated in the code
Define negative tests that validate these issues have been mitigated.
Perform these tests at whatever level is appropriate to identify any remaining vulnerabilities.
Typically performed at the integration / system level
Risk-based Testing
Security testing during implementation
48 © Copyright 2011 Coveros, Inc.. All rights reserved.
Top 25 Most Dangerous Software Errors
SQL Injection
OS Command Injection
Buffer Overflow
Cross site scripting
Missing authentication
Missing authorization
Hard-coded credentials
Missing encryption
Upload of dangerous files
Untrusted inputs in a security decision
Unnecessary privileges
Cross-site request forgery
Improper limitation of a restricted file path
Download of code without integrity checks
Risky crypto algorithms
Use of potentially dangerous functions
Security testing during implementation
8/20/2013
25
49 © Copyright 2011 Coveros, Inc.. All rights reserved.
Testing non-functional security requirements that span features within the system
Includes Web Application Security testing of any web-based interfaces
– Learn to read the output of these tools and understand how the vulnerabilities identified can be mitigated!
Often includes internal Penetration Testing type activities to “test like a hacker”
– Fuzzing
– Password crackers
– Network port scanners
– Dynamic input strings
Integration and Systems Testing
Security testing during implementation
50 © Copyright 2011 Coveros, Inc.. All rights reserved.
Web Application Scanners
Where to use? – Looking for XSS, Injection and input validation vulnerabilities; some
tools will attempt to actively exploit vulnerabilities.
Free Tools – Zap
– Nikto
– W3af
– Paros
– Skipfish
– Wfuzz
– ratproxy
Paid Tools – Netsparker
– WebSecurify
Tools to Support Web Security Testing
8/20/2013
26
51 © Copyright 2011 Coveros, Inc.. All rights reserved.
Zaproxy
Intercepting Proxy
Active scanner
Passive scanner
Brute Force scanner
Spider
Fuzzer
Port Scanner
Dynamic SSL certificates
API
Beanshell integration
52 © Copyright 2011 Coveros, Inc.. All rights reserved.
Engineer
IntelliJ IDEA/
Eclipse
subversion
JDepend
$$
Management
Hudson
Continuous Integration
8/20/2013
27
53 © Copyright 2011 Coveros, Inc.. All rights reserved.
Network Security Tools
Where to use? – Scanning for mis-configurations
– Testing for OS, application and network vulnerabilities
Free Tools – OpenVAS
Paid Tools – Nessus
– Core Impact
Tools to Support Penetration Testing
54 © Copyright 2011 Coveros, Inc.. All rights reserved.
Testing during the deployment process focuses on those tests that cannot be adequately completed within a development/QA environment plus any third party IV&V
– Red Team Penetration Testing
– Load and performance testing (for availability)
– Configuration testing
Red Team Penetration Testing is typically done by a team of security experts and includes both network and application testing
Overview
Security testing during deployment
8/20/2013
28
55 © Copyright 2011 Coveros, Inc.. All rights reserved.
Testing during maintenance and support focuses on: – Assuring that any identified vulnerabilities within the application,
supporting software, or network configuration are patched and revalidated
Based upon the identified vulnerability and patch, a wide variety of testing activities may be performed again to assure the patch operations properly and also does not break something else!
Overview
Security testing during maintenance / support
56 © Copyright 2011 Coveros, Inc.. All rights reserved.
Questions?
Contact Information:
Jeffery Payne
703.431.2920