I NDULGENC E

INDULGENCEThere is no need for oversight or management direction. All staff members

are superstars and act in the best interest of the company.

How to Audit Vulnerability Scans

Doug LandollCEO, Assero Security [email protected] (512) 633-8405http://twitter.com/douglandollwww.douglandoll.com

ISACA Phoenix Chapter Monthly Meeting - January

Agenda

Background – Security Risk Management & Assessments– Assessments as a process – Security risk management– Types of assessments

Anatomy of a Vulnerability Scan– Vulnerability Scan

Objective, Scope, and Execution Vulnerability Scan phases

How to Audit Vulnerability Scan (by phase) Checklist

Security Assessment as Process

Time

Changing Threats and Environment Increase Risk Over Time•New exploits•New system functions

•New regulations•Staff turnover

Security Improvements Lower Risk•Security awareness training•Security policy development•Operating system hardening

•Security patches•Anti-virus updates•Incident handling

Low

High

Risk

Security Risk Management Risk Assessment• threats / likelihood• vulnerabilities / exploitation• assets / impact• risk / countermeasures

Test & Review• scanning• audit of controls

Operational Security• patches• incident handling• training

Risk Mitigation• safeguard implementation• additional controls

Types of Assessments

Term Definition Purpose

Gap Assessment A review of security controls against a standard.

To provide a list of controls required to become compliant.

Compliance Audit

Verification that all required security controls are in place.

To attest to an organization’s compliance with a standard.

Security Audit A verification that specified security controls are in place.

To attest to an organization’s adherence to industry standards.

Penetration Testing

A methodical and planned attack on a system’s security controls.

To test the adequacy of security controls in place.

Vulnerability Scanning

An element of penetration testing that searches for obvious vulnerabilities.

To test for the existence of obvious vulnerabilities in the system’s security controls.

Types of Assessments Illustrated

Standard, Regulation

Controls

Assessments

Action List

Attestation

Gap Assessment Required

Compliance Audit

Covered

Effectiveness

Scoped

Security Risk Assessment

Risk & Recommendations

Security Audit

Selected

Anatomy of a Vulnerability Scan

Pre-Inspection• Define Scope• Define Objective• Define Project• Define Team

Footprint• Document IP ownership• Public Information Search• DNS Retrieval

Discovery• Open ports• OS fingerprint

Enumeration• General exploits

•open access, password guessing• Specific exploits

•Sendmail, DNS, SQL

Vulnerability Assessment False positive removal Severity rating Remediation advice

Report Generation• Introduction• Findings & Recommendations• Appendices

What controls were covered by the assessment?

Pre-Inspection: Scope

Control Areas:

– IP addresses (complete, internal/external)

– Web applications– Remote access– VOIP, Telephones– Wireless

Boundaries

– Physical boundary– Logical boundary– Outsourced functions– External interfaces– Relevant systems

Rigor

– Defined– Adequate

What were the boundaries of the assessment?

To what level of rigor was the assessment performed?

Scope: Physical Boundaries

Scope: Logical BoundariesExternal Interfaces

Scope: Level of Rigor

Low– Limited review, inspections, and tests.

Moderate– Substantial examination, inspections, and extended tests.

High– Comprehensive analysis, inspections, and extended depth

and scope of test

Document and communicate level of rigor through the adoption of a standard approach (e.g., NIST SP 800-53A, RIIOT, etc.)

Scope: Implications

Meeting scan objective

Scan caveats

Objective analysis of the effectiveness of current security controls that protect an organization’s assets.

If assessor believes the scope of the assessment is limited and may not meet the stated objective, the report should clearly indicate this.

Scoping: Limitations

Reasonable limitations– Common controls assessed elsewhere

Obtain report to ensure– Control limitations – sponsor does not control

other area Clearly indicate scope of assessment

Unreasonable limitations– Sever restrictions on rigor, methods, interfaces,

time, budget. Clearly state limitations in report Is it an adequate vulnerability scan?

Pre-Inspection: Objective

Objective Statement

– Defined– Frequency– Driver

Restrictions

– Reasonableness– Acceptance

Permissions

– Granted– DOS inclusion– Data modification

inclusion

What restrictions were placed on the assessment?

Were appropriate permissions granted?

Is the objective of the assessment clearly stated?

Pre-Inspection: Team

Independence– Claimed?– Adequate?

Expertise– Security expertise

Credentials (CISSP)– Audit expertise

Credentials (CISA)– Regulation / Business expertise (knowledge)

Was the team performing the assessment independent

and qualified?

Team: Objectivity

Who should perform the Vulnerability Scan?– Objectivity vs. independence– Budget and other factors affecting the decision

Footprint Audit Points

Pre-Inspection• Define Scope• Define Objective• Define Team

Footprint• Document IP ownership• Public Information Search• DNS Retrieval

Discovery• Open ports• OS fingerprint

Enumeration• General exploits

•open access, password guessing• Specific exploits

•Sendmail, DNS, SQL

Vulnerability Assessment False positive removal Severity rating Remediation advice

Report Generation• Introduction• Findings & Recommendations• Appendices

Footprint: IP Ownership

Did the assessment cover all the IP addressed identified by the system owner?

Did the assessment team independently verify the ownership of the IP addresses?

Were any of the identified IP addresses owned by a third party (i.e., hosting company), if so did the assessment team obtain permission?

Did the report clearly identify IP addresses not covered by the assessment (for example email server not covered for continuity reasons)?

Discovery Audit Points

Pre-Inspection• Define Scope• Define Objective• Define Team

Footprint• Document IP ownership• Public Information Search• DNS Retrieval

Discovery• Open ports• OS fingerprint

Enumeration• General exploits

•open access, password guessing• Specific exploits

•Sendmail, DNS, SQL

Vulnerability Assessment False positive removal Severity rating Remediation advice

Report Generation• Introduction• Findings & Recommendations• Appendices

Discovery: Discover Interfaces

Were interfaces within the boundary and scope completely discovered?– Did the assessor discover any additional

interfaces?– Did the assessment cover multiple protocols to

the same IP address? (ports?)– Did the assessment include:

VPN, IPS Web servers, application servers, custom apps DNS, mail servers

Discovery: Discover Information

Did the assessment team perform adequate analysis to discover information?– Public information (e.g. google hack)– Internal information (FTP, file shares)– Operating systems fingerprinted

Discovery: Complete Discover

Did the assessment team ensure complete discovery?– Load balancers– Virtual host (recent scan)– Wireless access points

Enumeration Audit Points

Pre-Inspection• Define Scope• Define Objective• Define Team

Footprint• Document IP ownership• Public Information Search• DNS Retrieval

Discovery• Open ports• OS fingerprint

Enumeration• General exploits

•open access, password guessing• Specific exploits

•Sendmail, DNS, SQL

Vulnerability Assessment False positive removal Severity rating Remediation advice

Report Generation• Introduction• Findings & Recommendations• Appendices

Enumeration: Determine Exploits

General exploits– Open access – no passwords– Password guessing and cracking

Specific exploits– Sendmail, DNS, SQL

Did the assessment team adequately determine exploits?

Vulnerability Assessment Audit Points

Pre-Inspection• Define Scope• Define Objective• Define Team

Footprint• Document IP ownership• Public Information Search• DNS Retrieval

Discovery• Open ports• OS fingerprint

Enumeration• General exploits

•open access, password guessing• Specific exploits

•Sendmail, DNS, SQL

Vulnerability Assessment False positive removal Severity rating Remediation advice

Report Generation• Introduction• Findings & Recommendations• Appendices

Vulnerability Assessment: Determine Impact

Did the team have a process for identifying and removing false positives?

Did the report utilize a ranking process for found vulnerabilities?

Was the security service (confidentiality, integrity, availability) affected indicated for each vulnerability?

Was there a re-test? Was the final scan free of “high” level vulnerabilities?

Report Audit Points

Report Generation• Introduction• Findings & Recommendations• Appendices

Pre-Inspection• Define Scope• Define Objective• Define Team

Footprint• Document IP ownership• Public Information Search• DNS Retrieval

Discovery• Open ports• OS fingerprint

Enumeration• General exploits

•open access, password guessing• Specific exploits

•Sendmail, DNS, SQL

Vulnerability Assessment• False positive removal• Severity rating• Remediation advice

Report: Introduction

Dates

– Report date. Recent?– Assessment date.

Consistent? Method

– Described adequately?– Meets rigor objective?– Meets compliance needs?

Findings & Remediation

– Each vulnerability Described Patch guidance Rated (impact) Ranked (order) Organized

– Rigorous enough to meet goals?

– Persistent findings?

Is the assessment recent and relevant?

Were the findings detailed, useful, and accurate?

Was the method used appropriate?

Report: Appendices

Start and Stop Times

– Match assessment date?– Adequate length?

Findings

– Match main report and summaries?

Remediation

– Match findings?

Do the start and stop times match the report?

Are the findings consistent?

Is there a remediation for each finding?

Checklist

See Handout