Dreaming of IoCs Adding Time Context to Threat Intelligence · PDF fileAdding Time Context to Threat Intelligence. ... Logstash Filtering. 21. ... Dreaming of IoCs Adding Time Context

SESSION ID:

#RSAC

Travis Smith

Dreaming of IoCsAdding Time Context to Threat Intelligence

AIR-W04

Senior Security Research EngineerTripwire, Inc.@MrTrav

#RSAC

@MrTrav2

THREAT INTELLongitude: -117.9190333Latitide: 33.8120584

#RSAC

@MrTrav

What is an Indicator of Compromise

3

An artifact observed on the network or operating system

#RSAC

@MrTrav

Formats

4

#RSAC

@MrTrav

What Is Threat Intelligence

5

“Evidence-based knowledge, including context, mechanisms, indicators, implications and actionable advice about an existing or emerging menace or hazard to assets that can be used to inform decisions regarding the subject’s response to that menace or hazard.”

“Intelligence is information that has been analyzed and refined so that it is useful to policymakers in making decisions.”

#RSAC

@MrTrav

CTI Triad

6

Actionable

#RSAC

@MrTrav7

THREAT INTELLongitude: -117.9190333Latitude: 33.8120584Culprit: Billy Two ToneAffiliations: SlingersVictims: ElderlyTactics: SlingshotTime: 1949

#RSAC

@MrTrav8

THREAT INTELLongitude: -117.9190333Latitude: 33.8120584Culprit: Billy Two ToneAffiliations: SlingersVictims: ElderlyTactics: SlingshotTime: 1949

#RSAC

@MrTrav9

Definition

#RSAC

@MrTrav

TAXII/STIX/CYBOX

10

Data Model• Package• Report• Campaign• Couse of Action• Exploit Target• Incident• Indicator• Threat Actor• TTP

http://stixproject.github.io/data-model/

#RSAC

@MrTrav

Sharing is Caring

11

Threat Intelligence / Information SharingAggregators of data sources

Open Source

Sandbox SolutionsWalled Gardens

Closed Source

#RSAC

@MrTrav

Aggregators

12

I know this is bad, do I see it?Search logs for hash/IP

I have something, is it bad?

Pros – proactive response

Cons – open source/free providers, questionable sanitization

#RSAC

@MrTrav

Walled Gardens

13

I have something, tell me what you think of itFind a file, reference it

See an IP, reference it

Pros – Sanitized and timely data

Cons Can be expensive

Performance - lots of lookups

#RSAC

Collective Intelligence Framework

#RSAC

@MrTrav

Collective Intelligence Framework

15

http://csirtgadgets.org/collective-intelligence-framework

https://github.com/csirtgadgets/massive-octo-spice

#RSAC

@MrTrav

Collective Intelligence Framework

16

RequirementsSmall: 16GB/8 cores/250GBLarge: 32GB/16 cores/500GBExtra Large: 64GB/32 cores/500GB

CIFv1 InstallationLots of dependencies, lots of effort

CIFv2 InstallationEasyButton!

#RSAC

@MrTrav

Collective Intelligence Framework

17

cif --otype ipv4 --format csvMD5

URL

FQDN

cif --otype ipv4 --format csvCSV

JSON

#RSAC

Logstash

#RSAC

@MrTrav

Intro to Logstash

#RSAC

@MrTrav

Intro to Logstash

20

INPUTS

FILTERS

OUTPUTS

FILE SYSLOG EVENTLOG STDIN 40+ More

GROK GEOIP TRANSLATE DATE 30+ More

ElasticSearch SYSLOG EMAIL STDOUT 50+ More

#RSAC

@MrTrav

Logstash Filtering

21

Utilizing Custom Patterns

GROK Message Filtering

Adding Custom Fields

Date Match

Using Translations for Threat Intelligence

#RSAC

@MrTrav

Logstash Filtering

22

filter { grok {

match => { "message" => "%{IP:client} %{WORD:method}

%{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}" }

} }

#RSAC

@MrTrav

Logstash Filtering

23

filter { grok {

patterns_dir => "/opt/logstash/custom_patterns"match => {

message => "%{123456}"}

} }

#RSAC

@MrTrav

Logstash Filtering

24

#RSAC

@MrTrav

Remove Capture Groups

Logstash Filtering

25

filter {if [message] =~ /^(([^,]+),([^,]+),([^,]+),([^,]+),...)/ {

grok { patterns_dir => "/opt/logstash/custom_patterns"match => {

message => "%{123456}"}

} }

}

(?<node_name>[^,]+),(?<node_type>[^,]+),(?<rule_name>[^,]+),(?<element_name>[^,]+),…

#RSAC

@MrTrav

Logstash Filtering

26

filter { if [message] =~ /^(([^,]+),([^,]+),([^,]+),([^,]+),...)/ {

grok { patterns_dir => "/opt/logstash/custom_patterns"match => {

message => "%{291001}“} add_field => [ "rule_id", “123456" ]add_field => [ "Device Type", “FIM" ]add_field => [ "Object", “File" ]add_field => [ "Action", “Modified" ]add_field => [ "Status", “Success" ]

} }

}

#RSAC

@MrTrav

Logstash Filtering

27

filter { ....all normalization code above here .... date {

match => [ "change_time", "M/d/YY h:m a" ]}

}

change_time: 3/2/16 10:20 AM

#RSAC

@MrTrav

Logstash Filtering

28

filter { ....all normalization code above here….translate {

field => “md5"destination => “maliciousMD5"dictionary_path => /opt/logstash/maliciousMD5.yaml'

}}

• Logstash will check the YAML for updates every 300 seconds• Configurable by adding refresh_interval => numSeconds

#RSAC

@MrTrav

Yet Another Python Script

29

cif –otype md5 --format csv

https://github.com/travisfsmith/iocdreaming

#RSAC

@MrTrav

Intro to Logstash

30

INPUTS

FILTERS

OUTPUTS

FILE SYSLOG EVENTLOG STDIN 40+ More

GROK GEOIP TRANSLATE DATE 30+ More

ElasticSearch SYSLOG EMAIL STDOUT 50+ More

#RSAC

@MrTrav

Logstash Filtering

31

Custom Fields:"Device Type" => "FIMDevice""Object" => "File""Action" => "Added""Status" => "Success"

Threat Intel Translations: "maliciousMD5" => "YES"

Date Matching: "change_time" => "3/2/16 10:20 AM""timestamp" => “2016-03-02T18:20:00.000Z"

#RSAC

@MrTrav

Logstash Filtering

32

1. Collect intelligence feeds

2. Update security tools with intel

3. Monitor observable which doesn’t match any feed

4. Feeds updated with observable previously already inspected….

#RSAC

TARDIS

#RSAC

@MrTrav

TARDIS

Threat Analysis, Reconnaissance, & Data Intelligence System

Historical Exploit/IOC Detection

Time Lord of Forensic Log Data

Available at: https://github.com/tripwire/tardis

#RSAC

@MrTrav

Yet Another Python Script

35

cif –otype md5 --format csv

https://github.com/travisfsmith/iocdreaming

#RSAC

@MrTrav

TARDIS

36

https://github.com/Tripwire/tardis

#RSAC

@MrTrav

TARDIS

37

1. Collect intelligence feeds

2. Update security tools with intel

3. Monitor observable which doesn’t match any feed

4. Feeds updated with observable previously already inspected….

5. Search repository for observable

#RSAC

@MrTrav

Architecture

maliciousMD5.yaml

#RSAC

Kibana Reporting

39

#RSAC

@MrTrav

Kibana

The ELK Stack

Search, Visualize, Dashboard

Zoom In & Out

#RSAC

@MrTrav

Kibana Searches

41

#RSAC

@MrTrav42

#RSAC

@MrTrav

Kibana Visualizations

43

#RSAC

@MrTrav

Area Chart

44

#RSAC

@MrTrav

Line Chart

45

#RSAC

@MrTrav

Pie Chart

46

#RSAC

@MrTrav

Geo Location

47

#RSAC

@MrTrav

Kibana Dashboard

48

#RSAC

@MrTrav49

#RSAC

Live Demo

50

#RSAC

@MrTrav

Notable Resources

https://github.com/tripwire/tardis

https://github.com/travisfsmith/iocdreaming

http://www.elastic.co

http://csirtgadgets.org/collective-intelligence-framework/

#RSAC

@MrTrav

Next Steps

52

0-3 MonthsIdentify Security Components

Which currently don’t integrate with Threat Intel?Which capture valuable observables?

3-6 MonthsIntegrate security tools with actionable threat intelligence

6+ MonthsFine tune workflows

SESSION ID:

#RSAC

Travis Smith

Dreaming of IoCsAdding Time Context to Threat Intelligence

AIR-W04

Senior Security Research EngineerTripwire, Inc.@MrTrav