Network Payload Analysis for Advanced Persistent …...Network Payload Analysis for Advanced Persistent Threats Charles Smutz, Lockheed Martin CIRT 2 About Speaker Name Charles Smutz

10000 7/6/2010 1

Network Payload Analysis for Advanced Persistent Threats

Charles Smutz, Lockheed Martin CIRT

2

About Speaker

Name Charles Smutz

Background Sysadmin, Networking, C&A

Current Job Lead Software Developer

Employer Lockheed Martin CIRT

Education Pursuing PhD at GMU

3

Background

• Understanding of APT– Persistent, Organized, Targeted CNE– Typical APT Attack Sequence

• Importance of Threat Focused CND/Security Intelligence

• You’ll have this by end of Summit

4

Topics

• Motivation– Why do network payload analysis

• Suggestions for Capabilities– What data to collect– Importance of Normalized Payload Analysis– Importance of Information Retrieval

• How to implement Capabilities– COTS/FOSS– Build Your Own

5

Why Network Analysis

• Important Data Source– 4n6 and Detection Intertwined

• 4n6 identifies and vets indicators• Detections feed 4n6

– Facilitate Pre-Compromise Detection– Strong Compliment to Host Analysis

• Complete Attack Sequence Analysis

6

Network Analysis Pros/Cons

• Benefits– Passive nature limits impact to network– Omniscience at network tap points– Control over data retention

• Drawbacks– Network forensics requires explicit data

retention– Encryption

7

Net vs. Host Compromise IRPredominately Host Predominately Network

Detection Malware C2 Beacon

Collection Host Logs, Memory Image, Disk Images

Network Logs, Packet Captures

Artifacts Malware, (Deleted) Toolsand Staged Data, Anything in Memory/Swap/Hyberfil• Commands• Passwords• Lateral Movement• Dropped Tools• Exfilled Data

Full Command and Control Decodes

• Commands• Passwords• Lateral Movement• Dropped Tools• Exfilled Data

DamageAssessment

Days/Weeks Hours/Days

8

Beyond FPC

• FPC is expensive, unwieldy

• Strategies for Targeted Data Collection– Network Transaction Logs– Payload Collection– Payload Metadata

• Information Retrieval For Accessibility

9

Network Transaction Logs

• Situational Awareness--Inbound of HTTP Requests– Direct Attacks (SQL injection etc)– Attacker Reconnaissance

• Options:– Sift through FPC– Collect, normalize, centralize all webserver logs– Snarf and reconstruct web activity

• Lots of tools to do this– Bro, Suricata, HTTPry, etc

• What about other protocols?

10

Attacks Moving Up Stack

http://www.sans.org/top-cyber-security-risks/

Document and Multimedia Viewers, Browsers

11

Attacks Moving Up Stack

Highly Targeted Social Engineering ExploitsUsers

12

Attacks Moving Up the Stack

From: [email protected]: from open.relay.com([10.10.10.10]) by mx.company.comReceived: from now.bad.com([172.16.1.1]) by mx.relay.comDate: Thu, 17 Jun 2010 12:03:41 -0700 (PDT)Message-Id: <1.1.2.3.5.8@mailer>X-Mailer: SillyMailer v3.14Subject: All your Base are belong to us

Please review attached.

Edward SpoofedSpoofed Inc.301-867‒5309

InfoKey: CreatorInfoValue: Acrobat PDF PrinterInfoKey: AuthorInfoValue: TK421InfoKey: ProducerInfoKey: ModDateInfoValue: D:20100616+08'00'PdfID1: 8d23f593e67be992ff3470dPdfID0: 798f9d8e3966ac586a61dc0

for(fqchp=0;fqchp<inxnh;fqchp++) {dnysj[fqchp]=dtkrx + hjnoa;}

if(ingmh){hsbsd();hsbsd();try {this.media.newPlayer(null);} catch(e) {}hsbsd();}

<Obfuscated Embedded Malware>

13

Attacks Moving up Stack

Layer Protocol Badness

EmbeddedObject

Application

Transport

Internet

Link

Email from legitimate email relay with Trojan Document Attachment

Ethernet -

IP -

TCP -

SMTP/MIME Spoofed Sender, SocialEngineering

PDF Exploit/Social Engineering,Malware

14

Indicators Moving Up Stack

Users

Use

ful I

ndic

ator

s

15

Indicators Moving Up the Stack

From: [email protected]: from open.relay.com([10.10.10.10]) by mx.company.comReceived: from now.bad.com([172.16.1.1]) by mx.relay.comDate: Thu, 17 Jun 2010 12:03:41 -0700 (PDT)Message-Id: <1.1.2.3.5.8@mailer>X-Mailer: SillyMailer v3.14Subject: All your Base are belong to us

Please review attached.

Edward SpoofedSpoofed Inc.301-867‒5309

InfoKey: CreatorInfoValue: Acrobat PDF PrinterInfoKey: AuthorInfoValue: TK421InfoKey: ProducerInfoKey: ModDateInfoValue: D:20100616+08'00'PdfID1: 8d23f593e67be992ff3470dPdfID0: 798f9d8e3966ac586a61dc0

for(fqchp=0;fqchp<inxnh;fqchp++) {dnysj[fqchp]=dtkrx + hjnoa;}

if(ingmh){hsbsd();hsbsd();try {this.media.newPlayer(null);} catch(e) {}hsbsd();}

<Obfuscated Embedded Malware>

12:03:31.165239 tcp 10.10.10.10.59170 -> 192.168.0.10.25 276 29770 FIN

16

Targeted Collection and Analysis

Email Web USB

Targeted attacks warrant targeted data collection

17

Email Data Collection Options

• Basic Email Transaction Data• Network Flow Data• Full Packet Capture• Normalized Emails

– Reassembled, Decoded, Indexed• Extended Email Metadata

– Headers: Subject, X-Mailer, Received– MIME Metadata: Names, Size, md5– Links

• Attachments (specific type?)• Attachment Metadata: Author, Creator, Dates

18

Usability Is Nice

19

Tiered Collection

Data RetentionLength

Size / Day Total Size

FPC (entire network) 1 week 1 TB 7 TB

Network Flow (entire network) 1 year 4 GB 1.5 TBStandard Mail Logs 2 year 50 MB 36 GB

Normalized, Indexed Emails 6 weeks 20 GB 800 GBExtended Email Metadata 6 months 500 MB 100 GB

Attachment Metadata 6 months 100 MB 20 GB

20

Accessibility Is Critical

• Rapid accessibility is critical:– Historical Detections– Identifying and vetting indicators

• Time to research an indicator matters– 1s, 1 minute, 1 hour, 1 day?

The faster you can research activity over large spans of time, the faster you’ll build threat intelligence

21

From: [email protected]: from open.relay1.com ([10.10.10.10]) by mx.company.comReceived: from now.bad.com ([172.16.1.1]) by mx.relay.comDate: Mon, 28 Dec 2009 5:48:02 +0800Message-Id: <1.1.2.3.5.8@mailer>X-Mailer: SillyMailer v3.14

<Malware 1.3>

From: [email protected]: from mx.openrelay2.com ([10.20.30.40]) by mx.company.comReceived: from now.bad.com ([192.168.2.2]) by mail.openrelay2.comDate: Mon, 5 Mar 2010 13:35:28 -0700 (PDT)Message-Id: <1.1.2.3.6.9@mailer>X-Mailer: SillyMailer v3.14

<Malware 2.0>

From: [email protected]: from relay.all.com ([10.70.50.60]) by mx.company.comReceived: from now.bad.com ([172.16.1.1]) by mx.relay.comDate: Thu, 17 Jun 2010 12:03:41 -0700 (PDT)Message-Id: <1.1.2.3.7.2@mailer>X-Mailer: SillyMailer v3.14

<Malware 2.01>

22

Ultra Light Weight Indexing

• Rapidly Search Key Indicator Types– IP addresses, Domains, etc

• Low Resolution– Log Type: proxy, email, etc– Time: ~Day– Per Device: proxy1, proxy2, proxy3

• Huge Scope– Time: indefinite retention– Data Sources: All

• Performance– Fast, << 1s response times

23

Ultra Light Weight Indexing

Data Type Source Date Indicatoremail-metadata mx1 2009-12-28 172.16.1.1inbound-http sensor1 2010-03-04 172.16.1.1email-metadata mx2 2010-06-17 172.16.1.1

Example search for 172.16.1.1:

24

Implementing Payload Analysis Tools

• Passive Collection:– Adapt an FPC

• Tail collection, filter normalize, extract – Adapt an IDS

• Filter, normalize, extract, archive• Inline Collection

– Milter, ICAP, etc

• Differences probably nuances, End goal is the same

25

Payload Analysis Issues

• Issues to be addressed:– Latency– Computational Expense– Implementing Payload Specific Capabilities

26

Payload Analysis: Latency

• IDS/IPS bound by real time• FPC provides on-demand data/processing

(arbitrarily long)• High Latency Analysis to be preformed (lookups)• Payload analysis for 4n6 usually should be

somewhere in between– Usually no benefit to be quicker than minute– For some applications slower than hour can

slow down response– Often daily processing makes sense

27

Payload Analysis: Complexity

• Expensive Tasks– Decoding, decompression, etc– Parsing, tokenizing, metadata extraction– Normalized archival (buffer copies)– Payload Identification– Any inherently computationally expensive

things• Statistical analysis• Compression• Etc

28

Latency and Complexity

• Heavy Buffering– 1 Gpbs * 60s = 7.5 GB RAM (dirt cheap)

• True Parallelism– Load balancing needs to move up stack also

• Example later

29

Implementing Payload Specific Capabilities

• Use existing network capabilities• Protocol Parsers

– HTTP::Parser, Mime::Parser, etc• Use payload capabilities• Payload Analyzers

– pdftk, pdf-parser, Officecat, etc• Use your in-house tools on extracted payloads

– Build network tools that work on objects (Abstraction)

30

Near Real Time IDS Platforms

• vortex (Lockheed Martin)– http://sourceforge.net/projects/vortex-ids/– Abstracts capture and TCP stream reassembly,

simple method for multithreading• snort-nrt (Sourcefire VRT)

– http://labs.snort.org/nrt/– Commitment to payload analysis

• Ruminate (George Mason University)– http://mason.gmu.edu/~csmutz/ruminate/– Focus on efficiency, scalability, completeness

of parsing

31

Vortex Overview

VortexStream Management, Flow Control

LibnidsTCP Stream Reassembly

LibpcapPacket Capture/Filtering

Cap

ture

d N

etw

ork

Traf

fic

File SystemSt

ream

D

ata

Analyzer ProgramReads Metadata, Loads Stream Data, Analyzes, optionally Purges

Stream Data

Stre

am M

etad

ata

(STD

OU

T)

32

Vortex Multithreaded

Vortex

Cap

ture

d N

etw

ork

Traf

fic

File SystemStre

am

Dat

a

Analyzer Program

Stre

am M

etad

ata

(STD

OU

T)

Xpip

esLo

ad B

alan

cing

Analyzer Program

Analyzer Program

Analyzer Program

33

Conclusions

• Network Data is important source for 4n6• Strategies for Network Data Collection

– Conventional (netflow, logs, FPC)– Targeted (playloads, payload metadata)

• Importance of data accessibility– Normalization– Search and Retrieval

• Ideas on Implementation

34

Questions?

[email protected]

Personal Blog:http://smusec.blogspot.com

35

36

APT Attack Sequence

Reconnaissance Weaponization Delivery Exploit Installation

Pre-Compromise

Post-Compromise

Reconnaissance Initial Intrusion

Establish Backdoor

Obtain User Credentials

Install Various Utilities

Priv. Escalation, Lateral Move.,

Data Exfil.

Maintain Persistance

Command & Control Actions on Intent