Best Practices in Product Security - vtt.fi · remediate! report! 3 Test results, logs and environment! variables sent to developers! 4 Developers run Fuzzers to validate, ... SCADA

© CODENOMICON Confidential and Proprietary. 1

Best Practices in Product Security

for ICT/ICS

Ari Takanen CTO, Codenomicon

9 November, 2010

DEFEND. THEN DEPLOY.


Introduction to the topic

  Traditionally only Hackers have used fuzzing to find bugs, and sometimes even reporting some of the findings

  More and more found vulnerabilities become traded zero-day exploits, and are never reported publicly (Stuxnet is a good example)

  As a solution, many product companies have started to integrate the same product security techniques to their software development •  Static analysis tools automate code auditing •  Fuzzing tools automate security testing

http://www.codenomicon.com/ sdl-fuzzing/

Location for SDL(C) related fuzzing materials:

code build/test

Developer Test team

1 New code submitted for build

2 Integrated code tested for security and robustness with Fuzzing suites

report remediate

3 Test results, logs and environment variables sent to developers

4 Developers run Fuzzers to validate, identify issues and remediate

Fuzzing Cycle: Agile or Waterfall


Typical Fuzzing Process

Input generation

Sending inputs

Target analysis

Identify interfaces

Exception analysis Report

Fitting the Fuzzing Process into the SDLC


Secure Development Lifecycle

7

http://msdn.microsoft.com/en-us/security/dd219581.aspx

© CODENOMICON Confidential and Proprietary.

Fuzzing and Robustness Testing

8


Security Vulnerability = Just A Bug


Real Results from ICS/SCADA?


What Is Fuzzing? - The Original Definition

  Just few years ago fuzzing was defined as:

  “Fuzz testing or fuzzing is a software testing technique that provides random data (fuzz) to the inputs of a program. If the program fails (for example, by crashing, or by failing built-in code assertions), the defects can be noted.”

  From: http://en.wikipedia.org/wiki/Fuzz_testing

11


Robustness Testing = Smart Fuzzing

  Robustness testing name used by ETSI and IEEE probably since early 80s

  Since 1999, we stole the term for our model based negative testing

  Also called: •  grammar testing •  syntax testing

  Based on sending systematically broken (rarely random) inputs to a software, in order to crash it

12


Fuzzing Coverage?


Key Questions in regards to Fuzzing

  Test Coverage: •  Q: Which interfaces and protocols are tested? •  A: Understanding of threat models and attack surface

are the most critical starting point to fuzzing

•  Q: How well something is tested? •  A1: Specification coverage explains how deep

knowledge the fuzzer has on a tested interface, and is typically represented by protocol models or protocol templates

•  A2: Anomaly coverage explains how well each protocol element is tested, and is the hardest metric to measure


Industrial Protocols (diverse set)

  SCADA specific: •  Modbus/IP •  Profinet/Profibus •  Ethernet/IP, DeviceNet, ControlNet •  DNP, DNPv3, ICCP •  UCA 2.0 and IEC61850 (including IEC 60870-5) •  Manufacturing Message Specification (MMS) •  OPC-UA (Binary and HTTP+SOAP+WSS)


More Industrial Protocols

  Traditional Internet protocols widely used everywhere: •  IPv4 (ARP, UDP, TCP, ...) •  FTP •  HTTP •  SNMP •  RPC (DCERPC, Sun-RPC) •  SMB •  STP •  NTP •  BOOTP •  LLDP (802.1AB Metro Ethernet) •  RSTP - Rapid Spanning Tree Protocol

  Then also wireless: WiFi, WiMAX


E2E Fuzzing: Fuzz the Entire System

  Fuzzing needs to be conducted for the entire system, hence end-to-end (E2E).

  Reduce down time by testing for unknowns

  Real-life-like test lab always needed


Why Fuzzing? Product Comparisons?

  Automotive industry (not any government) lead the design of crash test dummy •  Makes market comparisons

between vehicles meaningful

  Fuzzing “crash tests” software in similar manner •  In ICT, you are stuck with

what you get, there are no updates to software

•  Important to make market comparisons between software products

•  Don’t expect clean results


PROACTIVE SECURITY AND ROBUSTNESS SOLUTIONS

THANK YOU – QUESTIONS?

“Thrill to the excitement of the chase! Stalk bugs with care, methodology, and reason. Build traps for them.

.... Testers!

Break that software (as you must) and drive it to the ultimate

- but don’t enjoy the programmer’s pain.”

[from Boris Beizer]

Best Practices in Product Security - vtt.fi · remediate! report! 3 Test results, logs and environment! variables sent to developers! 4 Developers run Fuzzers to validate, ... SCADA

Documents