Metricon ‘06 Leading Indicators in Information security John Nye August 1, 2006

Metricon ‘06

Leading Indicators in Information security

John NyeAugust 1, 2006

2Symantec Security Services

Leading Indicators

In Medicine Body temperature

• Elevated values indicate probable illness and severity

• Temperature alone can not diagnose the illness

Characteristics Inexpensive to collect

Accurately diagnose the presence of the condition

May or may not reveal the nature of the condition

3Symantec Security Services

Leading Indicators in Information Security

Are there easily measured system attributes that predict an insecure configuration?

For example, does having a large number of open ports correlate to having an insecure environment?

Application

Evaluate an environment for its degree of vulnerability/risk to determine if additional investment is warranted (for example conducting a full vulnerability assessment)

4Symantec Security Services

Symantec Attack Center

5Symantec Security Services

SYMC Attack Center – The Data Set

Scans conducted between April, 2005 and July, 2006 Adoption of the tool has been increasing

Most scan results are relatively recent

449 Scans Conducted

Mostly External Penetration Tests

Nessus

Set Selection – We Eliminated: Suspected test scans (i.e. we were testing the AC, not a client)

Scans that weren’t used to produce a report

6Symantec Security Services

Methodology - Identifying Leading Indicators

Performed initial analysis using scans as the set

Vulnerability Score = sum of vulnerability severities divided by host count (calculated for each scan)

Scans ranked into quartiles based on vulnerability scores

Vulnerability Saturation = count of instances of a particular vulnerability divided by host count (calculated for each quartile)

Plotted each vulnerability’s saturation from quartile to quartile and examined the results

7Symantec Security Services

Eliminating Vulnerabilities as Potential Leading Indicators

Vulnerability eliminated from consideration if: Highest quartile saturation did not exceed 2%

Saturation didn’t increase with environment’s vulnerability

Particular to a type of environment, not generic to most environments (i.e. Web vulnerabilities)

Real Problems with the Data Set – 11th hour

Internal Network Scans Had to eliminate most vulnerable quartile completely from the

analysis because it contained multiple (and not-easily identified) scans conducted from within an enterprise perimeter

Probably eliminated several of the most vulnerable external scans in doing so

8Symantec Security Services

Findings (By Nessus Vuln ID)

All non-Web scanner findings with a final saturation > 2% identified during remote penetration tests.

Potential Leading Indicators

0

0.05

0.1

0.15

0.2

0.25

0.3

1 2 3

Quartile

Vu

lner

abil

ity

Sat

ura

tio

n

11951

11935

10092

10263

11002

11618

10114

11936

9Symantec Security Services

Top General Indicators

Leading Indicators (Preliminary Study)

0

0.05

0.1

0.15

0.2

0.25

0.3

1 2 3

Quartile

Vu

lner

abil

ity

Sat

ura

tio

n

Host Responds toSyn/Fin

ICMP TimestampRequest

OS Identified

10Symantec Security Services

Top Web Indicators

Leading Web Indicators (Preliminary Study)

0

0.1

0.2

0.3

0.4

0.5

0.6

1 2 3

Quartile

Vu

lner

abil

ity

Sat

ura

tio

n

SSL2.0

Web Mirror

Possible missing IISService Pack

HTTP Trace Enabled

HTTP: Does notreply with 404

HTTP DirectoryEnumeration

HTTP Server Typeand Version

HTTP Server Typeand Version

11Symantec Security Services

Correlation: Scans vs. Project ReportsLeading Indicators (Small Data Set)

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

1 2 3 4

Quartile

Vu

lner

abil

ity

Sat

ura

tio

n

FTP Banner (10092)

HTTP Server Typeand Version (10107)

ICMP TimestampRequest (10114)

HTTP DirectoryEnumeration (11032)

HTTP Trace Enabled(11213)

Possible Missing IISService Pack (11874)

•All data is from external penetration testsSmall sample spaceTop 8 general and top 8 Web vulnerabilities depicted (only 6 of the 16 were present in this data set.

12Symantec Security Services

Next Steps

Clean up the data set Quartile ranking of project reports doesn’t match that of Scans

Mix of internal and external scan data

Small sample set of project reports

Upgrade the math Statistical regression

Multi-vulnerability analysis

Repeat analysis for different types of environment Internal vs. External, Web vs. Generic, etc.

Implement the analysis directly in the Attack Center

13Symantec Security Services

Dangers with Leading Indicators

The leading indicator itself can not be used as a diagnosis

Gaming the system Administrators may attempt to resolve only those

vulnerabilities that are used as leading indicators.

14Symantec Security Services

Questions?

John Nye

Consulting Services Technical Lead

T. 617-768-2737

M. 617-501-3248

[email protected]

Thank You.