Monitoring ICS Communications

S4x15( Miami, FL ) www.Cri&calStack.com1

instrumenting and Monitoring ICS & Embedded Networks

Liam RandallCritical Stack

S4x14

http://www.criticalstack.com


Liam Randall – Blue Side

Liam Randall CEO, Critical Stack BS in Computer Science, Xavier University

Current Projects Incident Response Teach Bro ClassesRecon Detection Framework

Upcoming ConferencesJan, 2015- ICS

Bro Classes, Speaking?Feb, 2015 MAAWG

Bro ClassesJan, 2015- Shmoocon LABS

IDS Team, Bro ClassesJan, 2015 Flocon

Bro ClassesJan, 2015- Shmoocon Epilogue

Lab Team, Bro Classes

@Hectaman@CriticalStack

#S4x15



“The capital purchasing cycle and limited interface to ICS and embedded devices

represents a persistent and pervasive threat to all sizes of enterprises. Advanced

techniques and technologies are needed to address this threat.”

Bro Pla2orm

Executive Overview – What is our purpose



4 Exploits Field DataBackground

Current Techniques

2

Enforcement Sample Techniques

5

Overview ICS & Embedded

1

Bro Platform Overview

3

Monitoring Bro Approach

4

End Questions

6

Agenda – Briefing Overview



Internetof

Things

Device ManagementNetworks are now dominated by non-PC based devices.



0

12500

25000

37500

50000

2003 2010 2015 2020

Devices Population

62

Trends Against Us We are not only outnumbered the devices are growing in:

complexitycomputational powervariety

Lack of mgmt tools--> AV, HIDS, Update, Policy

Cisco IBSG

Growing DeviceManagement Gap

.08X 1.84X 3.47X 6.48X

Growth of Embedded Devices – We are on the wrong side of math



Capital Investments ICS, Embedded, Medical, Infrastructure is not easy to replaceand may be designed to run for 30+ years.

Embedded, TVs, mobile devices, gaming devices, packages...

Hardware Details Embedded Linux Dynamic Memory: 16- 64 MbFlash Memory: 16 - 128 Mb32 bit PowerPC

Protocols Sixnet, Modbus/TCP, DNP3ARP, UDP, ICMP, DHCP, PPP...

10/100 Ethernet 1 Port Primary ( 2 MACs )4 Port Switch

Communication Telemetry, Telephone (dialup,leased), radio...

RS232, RS485Multiple configurations

23

Sample Device – ICS Controller



Sony SNC-RZ30n PTZ Camera Sony cameras come in a large number of configurations.

Model appeared in 2003- similar to current models.

I/O Options 3 Alarm Inputs2 Alarm OutputsRS-232CRS-485

Protocols ARP, HTTP, FTP, SMTP, SNMP, DHCP, TCP/IP

10/100 Ethernet Optional WifiExpansion Slots

25x Optical Zoom Multiple Codecs, Frame Rates,etc.

SystemEmbedded Linux 8 MB of StorageExpansion Slots

Another Embedded Target – Similar Threat Surface



Devices – Network of things?



Security

Active Network Scanning(NESSUS / NMAP)

Patch Management Programs

End Users

Syslog

Anti Virus

HIDS: Host Based IDS

Host Based Firewalls

Signatures( Bad stuff we know about )

Flow Data

Segmentation- Air, VLANs

#fail

Traditional Techniques – Inadequate for Embedded / ICS



ICS FieldTraffic

Representative Attacks – Sample of compromises

Watering Hole Attack

CarnaBotnet

ICSRisks



ICS FieldTraffic

Real World SCADA AnomaliesFortune 20 Sample

Attack Scenario 1 – Unauthorized Access from Malicious Actor



Curious Anomalies The frequency this host is participating in the networkdoes not make sense.

Anomaly? 1 Time 1 Host1 Command7 Day Period

Examine Modbus CountAll Participants by Exception

Normal Comms Regular polling of data

23

Specialized Traffic Modbus – 7 Days of TrafficModified to Anonomize LocationActual Real World Incident from Aug 2013

Count Orig Resp Errors

1 10.67.4.147 10.18.226.13 - 6 10.1.1.35 10.72.230.36 GATEWAY_TARGET_FAILED_TO_RESPOND 18 10.1.1.35 10.60.30.73 ILLEGAL_FUNCTION 5189 10.1.1.35 10.60.30.73 ILLEGAL_DATA_ADDRESS123513 10.1.1.35 10.60.30.73 -164312 10.1.1.35 10.60.230.36 -



Watering Hole Attack

Leveraging Vulnerable Infrastructure Embedded devices may be turned against their operators.

Attack Scenario 2 – Demonstration from 10/13



1 Authenticate to device

Enable FTP: http://<IP>/command/ftpserver.cgi?FtpServerFunc=on

FTP: mkdir web\home

Upload resources

Install: http://<IP>/command/main.cgi?System=versionup

FAIL!

:)

2

3

4

5

Step 1:

Recon-

Default Creds

START

11

Sony SNC RZ30n – Firmware Update ProcessDemo- Deploying Malicious Payload to Clients



ICSRisks

Leveraging Vulnerable Infrastructure Embedded devices may be turned against their operators.

Attack Scenario 3 – Un-Recognized Risks



Vulnerability Overview Lot’s of vulnerabilities- this one is particularly bad.

CVE-2013-2802

EXPLOITABLE IMPACT ENVIRONMENTAL TEMPORAL

Access Vector

Attack Complexity

Authentication

Confidentiality

Integrity

AvailabilityImpact

Collateral Damage

% Vulnerable

Exploitability

Fix Available

Vulnerability Verified

Actual Score

10.0

9

CVS Scoring – CVE-2013-2802 Rank



Embedded Systems Systematic vulnerabilities can not be addressed ina vacuum- with in a system each component must besecured and monitored at numerous levels.

Host/OS Attack Attacker modifies firmware (OS) of device

- or -Attacker uploads/downloads malware

- or -Attacker maliciously reconfigures device

ICS Protocol Attack Attacker injects or modifies ICS logic

Connectivity DDOS, Man-in-the-Middle- availability effected

Network Comms Partners, controllers, or SCADA system itself maliciously modified

System AttacksHMI, Historian, Management systems attacked

8

3. ICS Threat Surface – Significantly Larger than discussed



ICSHoneypot

2013 TrendMicro ICS HoneypotRepresentative of real world conditions

Attack Scenario 3 – Who is attacking ICS systems?



Data Breakdown

Threat Classifica&on Reconnaissance- 100%Unauthorized Access- 77%Unauthorized Modification- 15%Information Disclosure- 69%Device Malware- 23%ICS Protocol- 15%

By the Numbers 18 Hours Until First Attacks 39 Documented Attacks12 Unique Targeted Attacks13 Repeated Attacks from Multiple Sources

Link www.trendmicro.com/cloud-content/us/pdfs/security-intelligence/white-papers/wp-whos-really-attacking-your-ics-equipment.pdf

3. TrendMicro ICS Honeypot – Threat type x GEO IP


http://www.jujubes-jelly-o.com


CarnaBotnet

Largest publicly known embedded wormaka “Alien Worm”

aka Internet Census 2012

Attack Scenario 4 – Global Embedded worm discovered by Bro Platform



Tracking Carna Botnet – The Team

Aashish Sharma Lawrence Berkeley National Lab Works with an incredible team of IR. Incredible speaker.

Bro Power User

Catch and Release with Bro System acts as an Internet Telescope

Sample of Anomalies June 2011- Morto Worm June 2012- “Alien Worm” June 2012- CVE-2012-2122-mysql-authentication-bypass

Link http://ee.lbl.govhttp://www.lbl.gov

Image 1 - Aashish Sharma


http://www.jujubes-jelly-o.com

S4x15( Miami, FL ) www.Cri&calStack.com

420,000 Devices

Scan StuffDefault

Credentials

23

Carna Botnet – ”Port scanning /0 using insecure embedded devices”

? ACCESS SCOPE PAYLOAD 25% /0

“..we discovered an amazing number of open embedded

devices on the Internet.

Many of them are based on Linux and allow login to

standard BusyBox with empty or default credentials.”

“..insecure devices are located basically everywhere on the Internet. They are not specific to one ISP or country.

So the problem of default or empty passwords is an

Internet and industry wide phenomenon.”

“The binary on the router was written in plain C. It was compiled for 9 different architectures using the

OpenWRT Buildroot.

In its latest and largest version this binary was

between 46 and 60 kb in size depending on the target

architecture.”

hJp://internetcensus2012.bitbucket.org/paper.html


http://internetcensus2012.bitbucket.org/paper.html


Carna Botnet– Lets look at the payload....

Directory Listing Compromised Device This is from one sample device- there would be minor differencesbetween the 9 different architectures.

Custom Payload 4 ARM BinariesRevision Jun 28, 2012Activity Back to May 30, 2012

“Hilinux” Busybox Linux (none) 2.6.24-rt1-hi3520v100

#2010033002 Wed Mar 31 13:05:50 EST

2010 armv6l unknown

Default Password root / <blank>root / 123456

Daemon tcp/210 https://isc.sans.edu/port.html?port=210

4K Payload Scanning filesLogs

-rwxr-xr-x 0 root root 8610 Jun 28 19:19 t2.arm_v6k-rwxr-xr-x 0 root root 13492 Jun 28 04:44 sp.arm_v6kdrwxr-xr-x 0 root root 0 Jul 23 2007 run/-rw-r--r-- 0 root root 33 Jun 28 04:02 response-rw-r--r-- 0 root root 371 Jun 28 04:02 readme-rw-r--r-- 0 root root 49152 Jul 5 09:19 pz-rw-r--r-- 0 root root 0 Jul 3 13:01 j-rw-r--r-- 0 root root 33 Jun 28 04:02 idhash-rwxr-xr-x 0 root root 5013 Jun 28 19:19 ht.arm_v6k-rw-r--r-- 0 root root 33 Jun 28 04:02 challenge-rwxr-xr-x 0 root root 10938 Jun 28 04:05 b.arm_v6k-rw-r--r-- 0 root root 10 Jul 3 13:21 1.run-rw-r--r-- 0 root root 10 Jul 3 13:21 0.run


https://isc.sans.edu/port.html?port=210


Device – What do the devices look like?

Dozens of Vulnerable Models Consider where in your network these resources would be deployed.- Sensitive area’s- Behind your firewall

One “Chinese” OEM Production traced by to single OEMInitially very concerning

Retailed By Meier Grocery StoreSams ClubAmazon.comCostco100’s of Retailers online

Link

https://www.q-see.com/http://wansview.net/

Image 1 - Vulnerable Wansview PTZ Camera Image 3 - Vulnerable Smarteye PTZ Camera

Image 2 - Vulnerable Q-See DVR


https://www.q-see.com/

http://wansview.net/


A Picture – is worth 420,000 devices....

Carna Botnet Details Most camera’s on Asian based networks.Scattered activity, single origin.SYN Packets Only

Top ASN (4134) = 25% of InfectionsASN 4134 (CN)- China Telcom

Top 5 ASN- 50% of Infections -ASN 3462 (TW)- Data Communications Business Group-ASN 4837 (CN)- China Unicom-ASN 9121 (TUR)- Turk Telcom-ASN 4788 (MY)- TM Net

Top 16 = 60% of Infections Long Tail of Infections Global in Scope

hJp://internetcensus2012.bitbucket.org/paper.html


http://internetcensus2012.bitbucket.org/paper.html


BroPlatform

OverviewCapabilities, use cases, and direction.



Bro – is short for Big Brother

Bro is three things ... The hardest part about Bro is that there are so many distinctuse cases for the Bro Platform

Turing Complete PLEvent on traffic, files, protocolsSyntactically like Python

Utilities to manage BroAPI, Intefaces, etc.

2

1

BroApps

BPLBro Programming Language

Bro Platform

Bro-IDS

Monitoring, Vulnerability Mgmt, DLP, Analysis, File Analysis

( Really just Bro Scripts )

3



Bro Platform – Dozens of use cases

Bro has use cases in.. Security, Monitoring, Reliability, Discovery, Compliance



Bro Functions – Three things Bro does

Protocol Logs Detailed protocol logs for each

network protocol; including logs fortunnels, IPv4/6, files and more

Alerts Bro-IDS is preconfigured with avariety of signature and anomaly

notifications

Actions Bro Programming Language is the real power;pivot to external applications, take advanced

protocol based decisions & more.







notifications



Devices

Servers

Tap: Bro

Sensor

Sensor Components







notifications


protocol based decisions & more.Ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service

Time string addr port addr port enum string

1355284742 AZIHpPIejvi 192.168.4.138 68 192.168.4.1 67 udp -

1326727285 K4xJ9AKH56g 192.168.4.148 55748 196.216.2.3 33117 tcp ftp-data

1326727283 Jd11tlLtlE 192.168.4.148 58838 196.216.2.3 21 tcp ftp

1326727287 bVQHYKEz2b4 192.168.4.148 54003 196.216.2.3 31093 tcp ftp-data

1326727286 5Dki82HwJDk 192.168.4.148 58840 196.216.2.3 21 tcp ftp

1355284761 YSJ6DDKEzGk 70.199.104.181 8391 192.168.4.20 443 tcp ssl

1355284791 BqLVVfmVO6d 70.199.104.181 8393 192.168.4.20 443 tcp ssl

1355284761 ya3SvH6ZxX4 70.199.104.181 8408 192.168.4.20 443 tcp ssl

1355284812 sxrPWDvcGQ2 192.168.4.20 48433 67.228.181.219 80 tcp http

1355284903 vlvQgRiHE54 192.168.4.20 14655 192.168.4.1 53 udp dns

1355284792 gn5FV4jeOJ4 70.199.104.181 8387 192.168.4.20 443 tcp ssl

1355285010 uEb3j6nYBS7 59.93.52.206 61027 192.168.4.20 25 tcp smtp

1326962278 SE2LJ7PLwIg 189.77.105.126 3 192.168.4.20 3 icmp -

1326962279 T6rMQFaMCie 95.165.30.73 3 192.168.4.20 3 icmp -

1329400936 qtNmAmHhDM4 192.168.4.20 14419 65.23.158.132 6668 tcp irc1329400884 cOctAcZusv2 192.168.4.20 32239 89.16.176.16 6666 tcp irc







notifications



#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto note

#types time string addr port addr port enum

1359673187 TLDtWBOrstk 192.168.0.120 61537 50.76.24.57 8443 tcp SSL::Invalid_Server_Cert

1359673187 L4bDTmPqvs2 192.168.1.8 49540 174.143.119.91 6697 tcp SSL::Invalid_Server_Cert

1359673187 JAvYksFW1Qb 207.188.131.2 5373 160.109.68.199 8081 tcp SSL::Invalid_Server_Cert

1359673188 - 192.168.0.57 62220 216.234.192.231 80 tcp Rogue_Access_Point

1359673188 5OYpDdtlnfd 192.168.0.147 45009 93.174.170.9 443 tcp SSL::Invalid_Server_Cert


1359673188 - - - - - - Software::Vulnerable_Version

1359673188 93CIvevOuxk 192.168.0.147 51897 98.136.223.39 8996 tcp SSL::Invalid_Server_Cert

1359673209 YpCOvC9p4Ef 208.89.42.50 48620 207.188.131.2 22 tcp SSH::Login

1359673210 SaKFGzmdXLl 207.188.131.2 11175 23.5.112.107 443 tcp SSL::Invalid_Server_Cert

1359673214 XLE8fYl5Tvg 207.188.131.2 11677 208.66.139.142 2145 tcp SSL::Invalid_Server_Cert


1359673218 NyPHd3qjIKe 208.89.42.50 43891 207.188.131.2 22 tcp SSH::Login

1359673223 0skn2N4oYbj 192.168.1.116 49249 15.201.49.137 80 tcp HTTP::MD5

1359673224 Q83ji8AFOO1 192.168.1.116 49250 15.192.45.26 80 tcp HTTP::MD5

1359673229 WU57HOSwkEj 208.89.42.50 62165 207.188.131.2 22 tcp SSH::Login







notifications



Devices

Servers

Tap: Bro

Sensor

Sensor Components Extracted File AnalysisSignature Analysis • Active Analysis! Malware Hash Registry • Intel Comparison ! OSINT, FS-ISAC, DOE CIRC…

Active Analysis • www.Malware-Tracker.com

• Static & Dynamic Analysis • Cuckoo Box? Volatility

Long Term Analysis • Coverage for Mobile Devices, Embedded • Post Compromise Research • Analysis- copy of every EXE in Company

Predicative Analysis • AV, Malwarebytes! Open a Ticket • Content Analysis- Keywords,

Files:



AtomicIntel

Network MonitoringAdvanced Atomic Intelligence Application



Terms & Definitions – Signature Detection vs. Anomaly Detection

Classically Speaking... In the literature you will typically find IDS’s broken into two distinctcategories- Signature or Anomaly based Detection.

Bro is designed to face Next Generation Challenges.

Signature Detection atomic indicators

domains, file hashes, IPv4/6Traditional Signatures

Algorithms

Anomaly Detection Traffic AnalysisFlow AnalysisProtocol Analysis

Bro Platform Hybrid System

Best of Both Worlds+ a programming language

Bro Deployment Today we concentrate on that


S4x15( Miami, FL ) www.Cri&calStack.com37 4

ICSI SSL Notary

Team CYRMUMalware Hash

Internal Feeds?

AlertsActionProtocol

OSINTAbuse.ch

MalwareDomain

List

SpamhausDrop

Bro Intelligence Framework – Actual Indicators

CRITs::Mul&ple_Campaign_Hits Recently 2 items on the zzAPT campaign were hit CRITs UIDs: 504f88abe0742e059a424144, 509697c6e0742e4d547a907d



Protocol Location Intel Type

IP Connection AddressDNS Request, Reply Address, DomainFile Hashes Generated HashFile Name Name

HTTP- HEADER HOST DomainHTTP- HEADER REFERER DomainHTTP- HEADER X-FORWARDED-FOR DomainHTTP- HEADER USER-AGENT SoftwareSMTP-HEADER FROM Domain

SSL / TLS X-509 Certificate CN Domain

.. exhaustive to list all the permutations!

Bro Intelligence Framework – More effective use of atomic indicators



Signature Evasion – Threat actors modify their TTPs to evade detection efforts

Each file, ip, domain, etc.. can be modified.Overly simplified example to communicate concept.

58

Signature User Agent = “DirBuster”

Evasion “User Agent = “DirBreaker”

Signature Effectiveness Despite their evadability signatures are still an effective weapon against particular types of

threats and threat actors.

More advanced threat actors are actively monitoring defensive TTPs, measuring attack success rates, and actively working to evade

detection efforts.

+ evasion



Socratic Ideal– Anomaly Detection

What should your network look like? You can not secure what you do not understand.

Green HTTP

Pink FTP-DATA

Red FTP

Payload Upload

Normal



ViewingICS &

Embedded

Network MonitoringDefending ICS & Embedded Systems

MoreBro

37



Payload Upload

Normal

$ less conn.log | bro-cut service|sort| uniq -c | sort -n

11 ftp 15 http 158 ftp-data

$ less conn.log | bro-cut service|sort| uniq -c | sort -n 14 http

Bro -‐ conn.log

38What should your network look like? You can not secure what you do not understand.

“Ground Truth” – A real record of communication



Whitelist or blacklist activity, behavior on your network? Bro gives you access to the internals of each protocol in real time as it happens.

Payload Upload

Normal

1 /command/all-configuration.cgi 1 /command/ftpserver.cgi 1 /command/main.cgi

11 /command/inquiry.cgi

1 /command/inquiry.cgi?inqjs=camctrlright 1 /command/ptzf.cgi?AreaZoom=94,35,158,62 2 /command/inquiry.cgi?inqjs=tvstandard 2 /command/ptzfctrlright/inquiry.cgi 3 /command/inquiry.cgi?inqjs=sysinfo 64 /command/ptzf.cgi

hJp.log URI’S

{ }

40

Deeper Inspection – Protocol and Payload Details



Know thyself: Part II You do need to have an understanding what normal means to you.

Normalhost device_type 58.107.168.125 Known::MODBUS_MASTER

58.107.168.121 Known::MODBUS_SLAVE

58.107.168.123 Known::MODBUS_MASTER

58.107.168.119 Known::MODBUS_SLAVE

58.107.168.121 Known::MODBUS_MASTER

modbus.log

Normal?

41

ICS Specific Protocols – Protocol and Payload Details



Know thyself: Part II You do need to have an understanding what normal means to you.

58.107.168.121 6350 53774 48652 0.515266 58.107.168.121 6352 8002 13124 0.515266 58.107.168.121 6354 16244 26487 0.515266 58.107.168.121 6368 52973 28967 0.515266 58.107.168.121 6370 14484 22486 0.515266 58.107.168.121 5020 8884 0 0.021755 58.107.168.121 5021 548 0 0.021755 58.107.168.121 5022 8840 0 0.021755

modbus_register_change.log

43

ICS Specific Protocols – Protocol record; what actually happened in the SCADA System.



BroPolicies

Bro PoliciesPinning embedded & ICS Behavior

MoreBro

44



2

46

const known_modbus: set[addr, ModbusDeviceType] &redef; global rogue_modbus: set[addr, ModbusDeviceType]&redef;

if ( [master, MODBUS_MASTER] !in known_modbus && [master, MODBUS_MASTER] !in rogue_modbus)

NOTICE([$note=Rogue_Modbus, $msg="Rogue modbus master detected", $sub="MODBUS_MASTER", $id=c$id]);

add rogue_modbus[slave, MODBUS_SLAVE];

Who? – Should be there?



known_modbus_pairs[58.107.168.123]= table(); add known_modbus_pairs[58.107.168.123][58.107.168.121];

add discovered_modbus_pairs[master][slave];

47

if (master in known_modbus_pairs && slave in known_modbus_pairs[master])

ICS Peer Groupings – Partner Pinning



add approved_comms[192.168.0.236, Analyzer::ANALYZER_HTTP]

if ([c$id$resp_h, atype] !in unapproved_comms) { add unapproved_comms[c$id$resp_h, atype];

Files::add_analyzer(f, Files::ANALYZER_EXTRACT);

Real Time Response – On Violation, Extract Files.



Questions?

?

55



Thank you! BYE!


Monitoring ICS Communications

Technology

embedded icss4x15 miami

s4x15s4x15 miami

purposes4x15 miami

maths4x15 miami

xgrowth of embedded

ics bro classes

gaming devices

mobile devices