Event Correlation: Security’s Holy Grail? Matthew Caldwell, CSO, GuardedNet BlackHat Briefings Las Vegas, NV ©2002 GuardedNet. All rights reserved. Confidential.

Event Correlation:Security’s Holy Grail?

Matthew Caldwell, CSO, GuardedNetBlackHat Briefings Las Vegas, NV

©2002 GuardedNet. All rights reserved.

Confidential – Do Not Copy or Distribute

August 1st 2002

Why turn to Correlation?

> Organizations have multi-vendor security infrastructures and can not integrate each vendor’s log data for complete threat assessment.

> No complete view of the enterprise’s security environment.

> Log data overload allows for intruders to bypass the watchman.

> High rate of false positives with present intrusion detection systems, caused by their orientation toward attack signatures or attack anomaly detection and not qualified threats

> Vendors that only manage their own products also fail to scale to even medium and large infrastructures.

> Limited security budgets.

Correlation and Security

> Correlation is particularly valuable to security teams, because information security is a many to one process, rather than a one to one or exception-based process.

> Network Management is a good example of an exception-based process. Correlation is much less valuable in that environment.

View of correlation in relation to other products

Security Hardware Devices

Security Event

Management (SEM)

Configuration & Control Product

i.e. Provider-1, Site Protector

Heterogeneous collection & correlation

threat analysis

Vendor-specific configuration & control

of point solutions

Vendor-specific point solutions

Ho

st IDS

Netw

ork

IDS

Firew

all

An

tivirus

Ap

plicatio

ns

Ro

uters

Servers

Correlation Pre-requisites

1. Data Transport – Moving Log Data from point A to point B•   Security of the data is important at this level, by

reducing risk exposure by encrypting and authenticating the data, data transport can be accomplished with high certainty.

• 2. Data Normalization – Taking the data from disparate log formats and creating a flexible single log format.

• 3. Aggregation – Taking Data from multiple B level devices and consolidating the data into one system.

Correlation Pre-requisites

• 4. Data Reduction – Reducing the data through deletion of duplicates, combining similar events into a single filtering, or using summary information. Compressing the data also is helpful to reduce bandwidth usage.

•  • 5. Correlation Engine – The process of correlating field

variables into a unique consolidated event.

What is Correlation?

• Cor·re·la·tion  n.

1. A causal, complementary, parallel, or reciprocal relationship, especially a structural, functional, or qualitative correspondence between two comparable entities: a correlation between drug abuse and crime.

2. Statistics. The simultaneous change in value of two numerically valued random variables: the positive correlation between cigarette smoking and the incidence of lung cancer; the negative correlation between age and normal vision.

Types of Correlation

Micro Level Correlation – Comparing fields within one data set. Most of the vendors today are performing this type of correlation. Micro correlation is typically done with just the fields in the normalized data. Otherwise known as atomic correlations.

Macro Level Correlation - Comparing multiple data sets. A few vendors are performing Macro correlation. Example: Correlating particular events from countries or comparing event names to vulnerability names.

Micro Correlation

Field Correlation - The ability to correlate specific events to a single field or multiple fields in the normalized data. Can be as simple as a search for all port 80 attacks.

Rule/Pattern Correlation – The ability via specific rules to label a certain set of events as a correlated event. Often used to correlate events over a long period of time.

Macro Correlation

Vulnerability Correlation – The process of mapping event IDS event names to vulnerability names this allows for reduction in false positives. This is particular useful with a good tuned vulnerability scanner and IDS.

Profile (Finger Print) Correlation - Forensic network data such as remote port scans, remote OS finger prints, finger information, and banner snatching provides a series of data sets that can be compared to help correlate attacks to attacker profiles.

Watch List Correlation – Using a set of learned inputs the watch list can serve as a reminder of previous offenders in real-time. Correlating previous attackers with current attacks.

Who Needs Correlation?

Anyone Managing and Monitoring over 2 Firewalls or Intrusion Detection Systems:

• Managed Security Providers• Corporate Enterprises • Federal/State/Local Governments• Critical Infrastructure• ISACS

So how does correlation reduce false positives?

Step through correlation – Packets are seen across a router ACL, attackers packets pass through firewall, miss the IDS, get picked up by the anomaly detection and the IIS-UNICODE exploit stops at our application security product or the correlations system compares the event type from the application security product with a vulnerability database. Well, no vulnerabilities existed, and the Server was Apache not IIS.

All of the data is seen and understood. The analyst and correlation system can make judgments based on knowing all the facts of the attempted intrusion. Will disregard, the attack as NIMDA.

Understanding Market Powwow

According to some vendors correlation = aggregation

Others would like you to believe that correlation is some fancy algorithm that puts everything together for you. Usually aimed at impressing the Boss with big words.

Some of them would like you to believe that correlation is a verb not a noun.

Beware of the Snake Oil’s (Really just a glorified Syslog viewer with color) and the Candle’s (Blow out Easily)

The Capital Players – Current Products Claiming Correlation

CyberWolf

eSecurity

GuardedNet

Intellitactics

ISS SiteProtector

NetForensics

OPEN

OpenSystems

What to do with Correlated Data?

Examples of what to do with Correlated Data  Threat Analysis – This is a process of taking the correlated data and

determining what it means and then prioritizing the data. Includes the reduction of False Positives.

  Forensic Analysis – The ability to analyze correlated events historically

for trending and for prosecution.   Policy Analysis – Correlation provides a holistic view, providing a

complete picture of your security posture.

Threat Analysis Process

Threat Analysis and Trending

Forensic Analysis

Holistic View: The High Ground

Benefits to the Security Organization

> Real time view of event data from all systems

> Centralized repository for event data from all systems

> Provides threat rankings based on severity, allowing the analyst to focus on true threat vs. false alarms

> Enhances ability to proactively respond to threat in the fastest time possible, with most complete information

> Increases efficiency of security operation, reducing costs

> Scales to monitor global networks

Why is Correlation and Threat Analysis Important?

> Reduce operating costs through efficiency and better resource allocation:• Monitor existing environment and deploy additional security measures without increasing

resources• Improve decision-making processes• Dramatically reduce response times

> Avoid the costs associated with a breach• Downtime, theft, or damage to reputation

> Maximize utilization of existing security infrastructure• Allows customers to manage their “Best of Breed” products and to truly get the full use

those resources

> Increase security awareness at all organizational levels

> Ensure better overall enterprise protection