Security Analytics for Data Discovery - Closing the SIEM Gap

Security Analytics for Data Discovery:Closing the SIEM Gap

Eric JohansenSr. Solutions Architect

[email protected]

Background

Virus CERT / Incident Response @ IBM

MSS Architect @ IBM Internet Security Systems SME: IBM SELM (Security Event & Log Management)

MSS Architect @ FishNet Security Launched Hosted SIEM and Co-managed SIEM Services

MSS Biz Dev @ Optiv

Overview

- Hunting- The SIEM Gap- The Problem with Hunting (and the Solution)- Unknowns (and how to turn them into

Knowns)- Wrap up

Hunting Defined

Proactive versus reactive approach to identifying incidents

Reactive: incident starts when a notification comes in.

Proactive: actively looking for incidents - based on patterns, intelligence, or even hunches.Source: Scott J. Roberts - http://sroberts.github.io/2015/04/14/ir-is-dead-long-live-ir/.

Hunting Maturity Model

3 Factors Contribute:1) Quality of data – the more data the better2) Tools provided to access and analyze the data3) Skills of the analysts using the data (hunting)

Source: David Bianco - http://detect-respond.blogspot.com.au/2015/10/a-simple-hunting-maturity-model.html.

Maturity Indicators:Threat Intel

Data Analysis ProceduresAutomation

Security Analytics – A Path to Hunting Maturity

“Advanced analytics are being integrated into security markets after rule- and signature-based prevention systems and tuning processes struggled to detect or stop most security breaches over the past few years”

Source: Gartner - The Fast-Evolving State of Security Analytics, 2016 – April 4, 2016.

The SIEM Gap Defined

Designed for the known- Normalize / parse logs with defined compatibility- Alerts based on policy - Pre-defined reporting- Automated Data Analysis (for compliance / audit)

If there’s not a rule, policy, report, or alert – nothing gets detected.

Architectural decisions made then - now fundamentally limit SIEM.(Technology advancements have enabled Security Analytics).

Not really designed for human interaction – i.e. hunting and incident response.

The SIEM Gap - Industry Analyst Perspectives

• Requires advanced skills and knowledge• Custom queries are difficult• Challenges collecting certain types of data• Lacks context for collected data• Too many false positive alerts

• Primary challenge is complexity• Performance limits galore• Data variety challenges• New environment explosion• Analysis? Where is that?

Data Analysis Evolution

Example Products

Delivery

Create Views

Use Cases

Predefined Reports

HP Arcsight

Vendors

Compliance

Structured Data Aggregation

Data Scientists

Visualize the Known

Custom Dashboards

SecurityOperations

Splunk

SMEs

Discover the Unknown

Security Analytics

Integrated Operations

Data Discovery Workflow

The Problem with Hunting

“Effective threat hunting remains the domain of the well-resourced, super-security-mature, extra-skilled security 1%-ers…”

Source: Anton Chuvakin – http://blogs.gartner.com/anton-chuvakin/2016/03/21/antons-favorite-threat-hunting-links/.

The Most Sophisticated Analytic on the Planet

A Profound Shift – Known to Unknown

Repor

t on answersCollect only

data required to answer

questions

Develop list of questions

Known

Analytics-enabled exploration and discovery

Collect everything

No list of questions

Cloud

Virtual

Unknown

Security Analytics – Techniques for the Unknown

Event Clusters

Rapidly analyze large data sets with machine learning – event clusters technology summarizes the data set based on commonality to allow for quick human analysis.

Security Analytics – Techniques for the Unknown

Association Analytics

Explore frequency in your data in different categories, i.e. IP addresses, geolocations, usernames, applications, etc.

Security Analytics – Techniques for the Unknown

Activity & Change

Compare datasets and timeframes for differences – trending up/down, what’s new, etc.

Security Analytics – Techniques for the Unknown

Cohort Analysis

“Guilt by association”

Security Analytics – Techniques for the Unknown

Visualization / Perspective

See the data – find outliers - explore

Security Analytics – Techniques for the Unknown

Natural Language Processing

Deconstruct messages to attempt to find the direct and implied informationcontent.

- Actions (verbs) – allow, deny, block, fail, etc.- Subjects (proper nouns) – addresses, usernames, etc.- Various other parts of speech (direct objects, prepositions, adjectives, etc.) that add nuance- Fuzzy

Security Analytics Search Engine- Much like Google – to the user Google looks like one big bucket of one big field.- Under the covers - adding in metadata to add hints and help improve relevance.

Security Analytics – Techniques for the Unknown

Clustering (Big Data) and Federation (Data Politics)

Security Analytics – Techniques for the Unknown

Flexible Real-time Data Collection

- Streaming Packet Capture: Forensic analysis on demand- Any TCP/UDP Port- All usual suspects (syslog, flat files, netflow, etc.)- Define repository, TTL, rate limit

Security Analytics – Techniques for the Unknown

Drag and Drop Import

- Simple browser interface to bring in disparate data- Define repository, TTL, delimiters, time (now versus time discovered in data)

- Take in anything human readable- Office files, Outlook PST, PDF, PCAP, configuration files, and much more.- Threat Intel and CMDB Data

Security Analytics – Techniques for the Unknown

Collaboration

- Pinboard- Save and share commonly used queries.

- Tags, Notes- Rapidly record observations in data

Security Analytics – Techniques for the Unknown

Automation

- Workflow- Create repeatable processes within your data.

- Remotes- Tie remote agent based actions into Workflow or use ad-hoc.

Security Analytics – A Path to Hunting Maturity

HuntingDiscover The Unknown

Rapid Event TriageDiscover The Cause

Incident ResponseDiscover Incident Context

Data AccessibilityDiscover From More Data

Search for outbound deny events and view clusters, trends and associations to spot high risk activity.

Drag log files from multiple sources into the system, retain original date, create time-correlated views.

Automatically correlate alerts and human data with automatically enriched infrastructure data.

Drag the 2G log file and 4G PCAP into the system as easy as uploading to Dropbox.

Clusters, comparisons and associations are automatic.

Hunting Maturity Model Revisited

3 Factors Contribute:1) Quality of data – the more data the better2) Tools provided to access and analyze the data3) Skills of the analysts using the data (hunting)

Source: David Bianco - http://detect-respond.blogspot.com.au/2015/10/a-simple-hunting-maturity-model.html.

Maturity Indicators:Threat Intel

Data Analysis ProceduresAutomation

Thank You

Eric JohansenSr. Solutions [email protected]