Transcript
Blending Automated and Manual Testing
Making Application Vulnerability Management Pay Dividends
My Background• Dan Cornell, founder and CTO of Denim
Group
• Software developer by background (Java, .NET, etc)
• OWASP San Antonio@danielcornell
My Background• Steve Springett, Application Security
Architect for Axway
• Software developer by background
• Leader of OWASP Dependency-‐Track
• Contributor to OWASP Dependency-‐Check@stevespringett
Goal: Continuous Security
• Prerequisites– Standardization– Continuous Integration– Continuous Delivery
• Compliments– Continuous Acceptance
Standardization
• All projects use same build system• All projects built the same way• Automated onboarding for new projects• Per-‐project build expertise not required
MetricsArtifacts
Continuous Integration
Continuous Integration Factory
Source Code (SCM)
Deliverables
Continuous Delivery
Continuous Delivery Factory
Artifacts
Security Metrics
Continuous Security
Continuous Security Factory
Source Code (SCM) Deliverables
Automated Security Metrics
• Static Analysis Findings• Dynamic Analysis Findings• Component Analysis Findings• Attack Surface Analysis Findings
Continuous Security Pipe
Jenkins CI ThreadFix Defect TrackerSCM
False Positive
TargetApplication
12
ThreadFixAccelerate Software Remediation
ThreadFix is a software vulnerability aggregation and management system that helps organizations aggregate vulnerability data, generate virtual patches, and interact with software defect tracking systems.
ThreadFix
• Open Source (MPL) application vulnerability management platform
• Create a consolidated view of your applications and vulnerabilities
• Prioritize application risk decisions based on data
• Translate vulnerabilities to developers in the tools they are already using
ThreadFix Community Edition• Main ThreadFixwebsite: www.threadfix.org
– General information, downloads
• ThreadFix GitHub site: www.github.com/denimgroup/threadfix– Code, issue tracking
• ThreadFix GitHubwiki: https://github.com/denimgroup/threadfix/wiki– Project documentation
• ThreadFix Google Group: https://groups.google.com/forum/?fromgroups#!forum/threadfix– Community support, general discussion
Vulnerability AggregationAutomated
Automated Manual
Access to Vulnerability Data
• Tradeoffs– The more places the vulnerability data lives, the more likely a compromise
–Withholding information from people who need it makes remediation more challenging
Managing All Vulnerability Data
• Manual activities– Penetration Testing– Code Reviews
• 3rd Party Data Sources– Customer-‐performed Testing– External auditor-‐performed Results
SSVL and Manual Results
• SSVL Data Format:– https://github.com/owasp/ssvl
• SSVL Conversion Tool:– https://github.com/denimgroup/threadfix/wiki/SSVL-‐Converter
RESTful API to Vulnerability Data
CustomR&D Monitoring
Dashboard
CustomDashboards
Key Performance Indicators
• Don’t go overboard – Use only what is needed• Progress and velocity• Per team comparison• Min/max/avg time to close per severity• By CWE
Lessons Learned
• Always automate static analysis• Always automate attack surface analysis• Always automate component analysis• Always automate dynamic analysis• Always perform manual dynamic analysis• Use native tools & workflow for static analysis
Lessons Learned
• Provide as much visibility as possible– Varying degrees of detail– Multiple delivery vehicles
• Set clear pass/fail criteria for Security Bars– Provide custom dashboard to provide status and advanced warning
Additional Advice
• Automation is not better than manual– It’s faster and more efficient– Both are necessary
• Don’t forget manual assessments– Threat Modeling– Secure Design/Architecture and Code Review– Penetration Testing
Finally
• Vulnerabilities in CI / CD / CS Infrastructure– Threat Model– Secure Architecture Review– Patch Management – Configuration Management– Key Management– Always use TLS
Q & A
top related