Ravi K. Iyer Information Trust Institute Coordinated Science Laboratory University of Illinois at Urbana-Champaign A Configurable Hardware Framework for.

Ravi K. Iyer

Information Trust InstituteCoordinated Science Laboratory

University of Illinois at Urbana-Champaign

A Configurable Hardware Framework for a Trusted Computing Base: Application to the Power Grid

1

2

Objectives

• Develop enabling technology to provide customizable level

of trust to a significant critical infrastructure as exemplified

by the Power Grid computing and communication systems.

• The focus is on design methods and runtime techniques to

achieve application-specific level of reliability and security,

while delivering optimal and timely performance.

Secure and ReliableComputing Base

TCIP- NSF Cyber Trust Center-Scale Project-Trustworthy Cyber Infrastructure for Power

www.iti.uiuc.edu Address technical challenges motivated by domain specific problems in

Ubiquitous exposed infrastructure

Real-time data monitoring and control

Wide area information coordination and information sharing

By developing science in

Trustworthy infrastructure for data collection and control

Wide-Area Trustworthy Information Exchange

Quantitative Validation

Present Day Power Grid Cyber Infrastructure

ControlArea

Coordinator

- 1000’s of RTU/IEDs- Monitor and control generation and transmission equipment

10’s of control areas feed data to coordinator

- State estimator creates model from RTU/IED data

- Peer coordinators may exchange information for broad model- Degree of sharing may change over time

Photos courtesy of John D. McDonald, KEMA Inc.

U.S. Power Grid Cyber Infrastructure Complexity

Copyright 2003 Edison Electric Institute. Source: POWERmap, © Platts, a Division of the McGraw Hill Companies.

CAISO

RTO WEST

ERCOT

MISO

TVA

GRID FLORIDA

GRID SOUTH

PJM

NYSO

ISO-NE

6

Increased Power Grid Trustworthiness via Secure and Reliable Computing Base

Level 2(Substation 1)

Control Center (EMS)

Control Center (EMS)

LAN

Level 3 (Enterprise)

Level 1(Sensors/Actuators)

LAN

IEDs

Vendor

ISO

Substation 2

Substation 4

Substation 3meter

Gateway

Application Specific Architectures

Customizable Reconfiguration

New Types of Platforms

Approach

• Explore processor/OS/Application level solutions to achieve low-cost, high-performance, scalable security and reliability checking in the same framework

• Provide small footprint solutions that not require large amount of extra hardware or software

• Provide solutions that can coexist with the current generation of critical infrastructure

• Ensure timely detection and recovery to prevent loss of service or damage to critical infrastructure

7

Approach

Vision:• Systematically transform the

computing base • for application-level security

and reliability

Main idea: • Derive application-centric

checks • embed them in the HW• access them with OS/middleware

support• validate them in power-grid

cyber infrastructureConsidering:• Both COTS and new architectures• technical challenges raised by

deployment/management

Current Generation of Low-end Devices (1)

• New generation devices for substation automation: Network Terminal Units (NTUs)

– Combines traditional RTU (remote terminal unit) functionality with superior processing power and advanced tools for data collection and control.

• Capabilities

– innovative data management

– easy-to-use configuration program,

• employs Windows interface and drag-and-drop database editing

– preservation of legacy components, subsystems, and data concentration applications

• assemble multiple databases from the available data points.

• ability to monitor real-time transfer of raw data to and from your NTU

– support for reconfiguration or expansion for changes and additions

– grow from a basic site to one with many IEDs,

• IED configuration and on- site setup

– upgrade of RTUs to NTU capabilities with minimal difficulty.

OpEn Connect NTU-7500

Advanced Control Systems

Current Generation of Low-end Devices (2)

• NTU-Substation Controller– high-performance– large database capacity – data concentrator and protocol converter applications – ability to process a large amount of data from IEDs, – interface a large number of discrete data acquisition and control devices in the substation. – use of virtual RTUs

• sort incoming IED and RTU data into discrete databases• can be based on data type, e.g., critical and non-critical• can configure Virtual RTUs to provide appropriate data subsets

• Design Features– distributed processing architecture; – multiple 32-bit microprocessors, – linked using a peer-to-peer type network– multiple IED isolated serial communication interfaces

• Operating Systems– IED: RTOS, e.g., Thread X– NTU/RTU: Linux, XP

OS

Middleware

Applications

Hardware

Fra

me

wo

rk I

nte

rface F

abric

Pipeline

Modules

Achieving Secure and Reliable Computing Base

Reliability and Security Microkernel (RSM)

Hardware

OS

Middleware

Applications

OS

Middleware

Applications

Hardware

Fra

me

wo

rk I

nte

rfa

ce

Fa

bri

c

Pipeline

Modules

OS

Middleware

Applications

Hardware

Reliability and Security Engine (RSE)

Reconfigurable processor-level hardware framework to support security and reliability Current features

On-core approach Framework and modules implemented on an FPGA

Available modules Malicious attack detection - Pointer taintedness detection - Information-flow signatures Transparent hang/crash detection for OS and applications

Reconfigurable operating system-level kernel module to support OS/application aware security and reliability services Current features

Two level hierarchy: - low-level pins interfacing with OS and hardware - high-level modules providing application-specific security and reliability techniques

Available modules Application/OS hang/crash detection Transparent application checkpoint

Automated Design Flow

ApplicationSource Code

Dynamic ExecutionProfile

Application code annotated with critical variables

Application code instrumentedwith interface to invoke H/W checks

Path-trackingstate machines

CheckingExpressions

General-purpose Processor

Reliability and Security Engine(RSE)

RSEInter-face

profiling Fanouts analysis

Regular compiler VHDL Translation & synthesis

Reliability Augmented Compiler

Path Tracking

Static-Checking

Register File

PATH_CHECK Instruction Committed

Write Buffer

EXPR_CHECKInstruction Committed

Main Memory

Runtime PathDLX Superscalarwith RSE

Error Detected

Static-Detector Module

Hardware Implementation: RSE Module

14

Security Partitioned Applications

Intelligent Electronic Device - SEL 3351 Data Aggregator

Main Processor

Secure CoprocessorProtected with RSE

Database Application

Streaming Video

DistributionApplication

AccessControl

Functions

Secured CodeKernelsSecure

Data

15

Example of Security Partitioned Applications

Intelligent Electronic Device - SEL 3351 Data Aggregator

Main Processor

Secure CoprocessorProtected with RSE

Database Application

Streaming Video

DistributionApplication

AccessControl

Functions

Secured CodeKernelsSecure

Data

Main application runs on COTS processor.

16

Security Partitioned Applications

Intelligent Electronic Device - SEL 3351 Data Aggregator

Main Processor

Secure CoprocessorProtected with RSE

Database Application

Streaming Video

DistributionApplication

AccessControl

Functions

Secured CodeKernelsSecure

Data

Coprocessor allows utilization of custom checking hardware.

17

Security Partitioned Applications

Intelligent Electronic Device - SEL 3351 Data Aggregator

Main Processor

Secure CoprocessorProtected with RSE

Database Application

Streaming Video

DistributionApplication

AccessControl

Functions

Secured CodeKernelsSecure

Data

Secure kernels

provide high levels of trust

18

Security Partitioned Applications

Intelligent Electronic Device - SEL 3351 Data Aggregator

Main Processor

Secure CoprocessorProtected with RSE

Database Application

Streaming Video

DistributionApplication

AccessControl

Functions

Secured CodeKernelsSecure

Data

Clearly Defined Interfaces Support

Development

19

Security Partitioned Applications

Intelligent Electronic Device - SEL 3351 Data Aggregator

Main Processor

Secure CoprocessorProtected with RSE

Database Application

Streaming Video

DistributionApplication

AccessControl

Functions

Secured CodeKernelsSecure

DataSecured DataRemains onCoprocessor

20

Coprocessor Integration within the Testbed

• Augment SEL 3351 with FPGA-based coprocessor.

• Nallatech FPGA Card available in PC-104+

• Demonstrate Coprocessor in various applications:– Undervoltage Relay– Video Streaming Distribution

FPGACoprocessor

PC-104+Computer

SEL

SEL 3351 Data Aggregator

21

Applications

Undervoltage Load Shedding Relay:• Monitor critical power data and take

corrective action before system is affected.

• Protect against accidental and malicious data corruption using RSE.

• Integrate FPGA coprocessor into SEL-3351 Data Aggregator using PC-104+ interface.

• FPGA Coprocessor with RSE will execute security critical kernels within protected application with a high degree of Trust.

Streaming Video Distribution:• Provide protection for distributed

distribution of Substation security camera video.

• Prevent malicious tampering with video feed due to bandwidth strangling by limiting each individual users bandwidth usage.

• RSE FPGA Coprocessor will protect small security critical kernels within the streaming application.

NallatechDIME-II withXilinx FPGA

SchweitzerSEL-3351

Data Aggregator

SchweitzerSEL-421

Relay

Xilinx JTAGDebug Cable

TCIP Testbed:Synchrophaser RelayProtected with RSE

Coprocessor Integration within the Testbed

Backups

23

Approach

Vision:• Systematically transform the

computing base • for application-level security

and reliability

Main idea: • Derive application-centric

checks • embed them in the HW• access them with OS/middleware

support• validate them in power-grid

cyber infrastructureConsidering:• Both COTS and new architectures• technical challenges raised by

deployment/management

Research and Engineering Issues

Application-Oriented Threats

General Platform Threats

Exploring HW support and use cases for “Tunable Trust”•Between components of substation•Between RTUs and the rest of the systemGoal:•Restrictions in bandwidth and processing power prevent security from being applied in current system

Setting up testbed to experiment with emerging hardware approaches:•Physical security attacksGoal:•Increase assurance that IEDs/RTUs can maintain secrets despite physical attack•Explore ways to update crypto and sw for devices that must last decades, unmanned

HW mechanisms to enable trustworthy sharing of computation on private data•“Tiny Trusted Third Parties”•Draws on secure coprocessing, oblivious RAM, blinded circuits•Prototyped on IBM 4758•Implementing smaller, cheaper, faster version on FPGAs•Application-aware security and reliability services

Setting up testbed to experiment with emerging hardware/software approaches:•RSE (reliability and security engine)•AEGIS•RSM (reliability and security microkernel)•MulticoreGoal: •Increase availability and resilience for general-purpose platforms exposed to network attacks

Su

bsta

tio

ns

Con

trol C

en

ters

an

d a

bove

Automated Design Flow

ApplicationSource Code

Dynamic ExecutionProfile

Application code annotated with critical variables

Application code instrumentedwith interface to invoke H/W checks

Path-trackingstate machines

CheckingExpressions

General-purpose Processor

Reliability and Security Engine(RSE)

RSEInter-face

profiling Fanouts analysis

Regular compiler VHDL Translation & synthesis

Reliability Augmented Compiler

Path Tracking

Static-Checking

Register File

PATH_CHECK Instruction Committed

Write Buffer

EXPR_CHECKInstruction Committed

Main Memory

Runtime PathDLX Superscalarwith RSE

Error Detected

Static-Detector Module

Hardware Implementation: RSE Module