hume.ictas.vt

http://www.hume.ictas.vt.edu

Smart Grid: Where Computation, Communication and Power Systems Meet

Sandeep K. [email protected]

with Hua Lin, Yi Deng, James Thorp, Lamine Mili

This work was partially supported by NSF grant EFRI-0835879 & an NSF IUCRC - S2ERC Project

About ACM

ACM, the Association for Computing Machinery is the world’s largest educational and scientific computing society, uniting educators, researchers and

professionals to inspire dialogue, share resources and address the field’s challenges.

ACM strengthens the computing profession’s collective voice through strong leadership, promotion of the highest standards, and recognition of technical

excellence.

ACM supports the professional growth of its members by providing opportunities for life-long learning, career development, and professional

networking.

With over 100,000 members from over 100 countries, ACM works to advance computing as a science and a profession. www.acm.org

The Distinguished Speakers Program

is made possible by

For additional information, please visit http://dsp.acm.org/

Outline

• Motivation– Need for Infrastructure Interdependence Study– Power System & Computing/communication – Smart Grid

• Need for Co-Simulation • GECO – Our Co-simulator• Designing New Relaying Scheme with GECO• All PMU-State Estimator with GECO

– Experimental Framework– Experimental Results and Interpretations

• Conclusions

Infrastructure Interdependencies

“Our nation’s infrastructures have becomeincreasingly interconnected and interdependent

… this creates an increased possibility that a ratherminor and routine disturbance can cascade into a

regional outage

… it also creates new assurance challenges thatcan only be met by a partnership between owners

and operators and government at all levels.”

President’s Commission onCritical Infrastructure Protection 1997

Examples of Critical Infrastructures

• Energy (electric power, oil, natural gas)• Telecommunications• Transportation• Water systems• Banking and finance• Emergency services• Government services• Agriculture• Others

* CMU SEI Study

What is “Power System”

8

Generation

9

renewable coal

natural gas nuclear

Transmission

10

substation

substation

substation

substation

substation

substation

substation

power tower

power tower

power tower

power tower

power tower

power pole

power pole

Distribution

11

residential

residential

residential

residential

residential

industrial

industrial

What is “Smart Grid”

12

http://www.elp.com/index/display/article-display/0045209435/articles/utility-products/volume-7/issue-7/product-focus/test-__measurement/measurement-tools.html

Smart Grid Vision

• Generation:– Micro-grid – Renewable energy– Gas turbines

• Transmission:– Wide area monitoring – Wide area protection and control– Real-time state estimation

• Distribution Level:– Smart metering– Demand response– Self-healing distribution network

13

Communication Infrastructure

14

Communication Techniques• Communication Link

– Telephone– Microwave– Co-axial– Fiber– Power line communication

• Communication Network– LAN– WAN– MAN– WLAN

15

A Wide Area Measurement Scenario

16

Control Center

Motivation• Smarter Grid entails more Cyber components

• Wide area measurement and Control • Communication Infrastructure• New Cyber Security Vulnerabilities

• Smart Grid is a Extremely Large Scale Cyber Physical System• ELCPS• Physical Dynamics controlled by Cyber Networked Control • Attack on the networked control can lead to disastrous Physical Dynamics

• Need to Study ELCPS• Too large for Analytical Study• Scalable but Accurate Co-Simulation is needed

• Need for co-simulation tools• Leveraging Existing Scalable Tools

• Study Wide Area Control issues but Security is Extremely Important to Study

Co-Simulation for CPS

18

Power System Simulation

Cyber & Network Simulation

To design a CPS system, engineers need tools to explore possible architectures, protocols, and configurations.

Smart Grid engineers should be able to precisely model the power system and the communication network together so that the system behaviors can be suitably predicted.

Synchronization

Other Power System/Cyber Co-Simulators

• EPOCHS: PSLF + NS2 [Cornell] • DEVS method: adevs + NS2 [ORNL]• PowerWorld + RINSE [UIUC]• PowerWorld + OPNET [UIUC]• PowerWorld + NS3 [Ga Tech] • OPNET extension [Jia Tong]

19

[1] K. Hopkinson, X. Wang, R. Giovanini, J. Thorp, K. Birman, and D. Coury. Epochs: a platform for agent-based electric power and communication simulation built from commercial off-the-shelf components.[2] J. Nutaro, P. T. Kuruganti, L. Miller, S. Mullen, and M. Shankar. Integrated hybridsimulation of electric power and communications systems. In Proc. IEEE Power Engineering Society General Meeting, pages 1–8, 2007.[3] C. M. Davis, J. E. Tate, H. Okhravi, C. Grier, T. J. Overbye, and D. Nicol. Scada cybersecurity testbed development. In Proc. 38th North American Power Symp. NAPS 2006, pages 483–488, 2006.[4] D. C. T. C. Malaz Mallouhi, Youssif Al-Nashif and S. Hariri. A testbed for analyzing security of scada control systems (tasscs). In Second IEEE PES Innovative Smart Grid Technologies Conference, 2011.[5] X. Tong. The co-simulation extending for wide-area communication networks in power system. In Proc. Asia-Pacific Power and Energy Engineering Conf. (APPEEC), pages 1–4, 2010.

Continuous Time System Simulation

• Discretize differential equations and time

20

Power System Dynamic Simulation

21

Initialize all state variables

Calculate state variable derivatives

Calculate secondary variables

Calculate network boundary variables

Integration step

t0

t=t+Δt

one

roun

d

t ………………

t0

A simulation round

Discrete Event System Simulation

• Occurrence of events are not uniform• Event-Driven

– Scheduler– Event Queue– Event Processing

22

Communication Network Simulation

23

1 2

4

3

Synchronization with errors in EPOCHS

24

event 1

event 2

t

………………

event 3

event 4

Stands for a round of power system dynamic simulation

Stands for a communication network event

Start Synchronization Point 1

Synchronization Point 2

t

event 5

event 6

………………

………………

X

X

Error 1

Error 2Power

Communication

Global Event-Driven Synchronization

25

event 1

event 2 ………………

event 3

event 4

Stands for a round of power system dynamic simulation

Stands for a communication network event

Start

t

event 5

event 6

………………

√

√

event 1

event 2

event 3

event 4 ………………

Global Event Queue

Power

Communication

Implementation of the Co-simulation Framework GECO

• PSLF– Power system– Written in

Java– Script: EPCL

26

• NS2– Communication

network– Written in C++– Script: OTcl

Co-Simulation Platform Structure

27

BasicModel

DynamicModel

PSLF Simulation

PSLFInterface

PowerCommunication

Protocols

NS2 Simulation

NS2Interface

PowerApplications

PowerApplications

…………

………………

………………

GlobalScheduler

GlobalEvent List

GECO To Study All PMU linear state estimator

• Global Event-driven Co-simulation

Power System Models

PSLF Simulation

PSLFInterface

PDC Applications

NS2 Simulation

NS2Interface

Super PDC Applications

PMUApplications

…………

………………

………………

GlobalScheduler

GlobalEvent List

State Estimation

MatrixInterface

Linear State Estimator

Internal Data Transfer External Data Transfer

Power System Protection• Relays protect power systems when faults happen

– Over current– Over voltage– Directional– Distance (Impedance)– Differential– Pilot

29

Distance Relay Protection Zones

• Primary: Zone 1• Backup: Zone 2, Zone 3• Time-delayed manner for backups: Zone 2(300ms), Zone 3(1s)

30

Problems with Backup Relays• Drawbacks

– Long waiting time– Over sensitivity– Hidden failures

• However, zone 3 is still needed

31

[1] S. Protection and C. T. Force. Rationale for the use of local and remote (zone 3) protective relaying backup systems. Technical report, North American Electric Reliability Council, 2005.

Network-based Backup Relay Protection

• Backup distance relays proactively communicate with other relays to obtain wider system visibility and make global protection decision– Software agents take control– Supervisory (master - slave)– Ad-hoc (peer - peer)

32

Supervisory Protection Scheme

33

Supervisory Scheme Operation (Slave)

34

Supervisory Scheme Operation (Master)

35

Ad-Hoc Protection Scheme

36

Ad-Hoc Scheme Operation (Peer)

37

Relay Searching

• Find the responsible relay group

38

Searching Algorithm

39

Decision Making• Decision is made by “OR” manner voting• Upper and lower time threshold

40

Co-Simulation Settings• New England 39-bus system• Communication network share same topology with

power system• 100Mbps bandwidth and 3ms latency for each

communication link• Without background traffic

41

Supervisory Protection on 39-bus System (Case 1)

42

Robustness against primary failure

43

AM

S

Supervisory Protection on 39-bus System (Case 2)

44

Robustness against hidden failure

45

AM

S

Supervisory Protection Communication Delay

46

Relay Agent ID

Supervisory Protection Communication Delay Analysis

47

Ad-hoc Protection Communication Delay

48

Relay Agent ID

Supervisory Protection with Link Failure

49

Relay Agent ID

Supervisory Protection Delay with Link Failure

50

Ad-hoc Protection with Link Failure

51

Relay Agent ID

Comparison• Real system implementation

– Supervisory: extra master agent needed– Ad-hoc: peer relays store system information locally– Hybrid mode

• Reaction time– Supervisory: long, uneven– Ad-hoc: short, even

• Robustness to network failures– Supervisory: increase by 20%-100%– Ad-hoc: increase by multiple times

52

Outline

• Motivation• Need for Co-Simulation • GECO – Our Co-simulator• Relay Case Study• All PMU-State Estimator

– Experimental Framework– Experimental Results and Interpretations

• Conclusions

Power System State Estimation

• Conventional– Slow scanning rate– Power injection, power flow, voltage magnitude– Non-linear, iterative solution

• All-PMU– 30 times/sec– Complex voltage and current– Linear, non-iterative solution

Cyber Security Considerations

• All-PMU state estimation is superior than conventional ones.

• But it can still be vulnerable to cyber attacks or network failures.– Intranet not completely safe– Many conceivable threat models

WAMS Infrastructure

56

Timer to catch up measurement rate

Outline

• Motivation• Need for Co-Simulation • GECO – Our Co-simulator• Relay Case Study• All PMU-State Estimator

– Experimental Framework– Experimental Results and Interpretations

• Conclusions

New England 39-bus System

PDC1

SPDC

PDC2

PDC3 PDC4

58

Area 1

Area 2

Area 3

Area 4

Co-Simulation Settings

59

[1] Kun Zhu, M. Chenine, and L. Nordstrom. ICT architecture impact on wide area monitoring and control systems’ reliability. 26(4):2801–2808, 2011.

[1]

[1]

Outline

• Motivation• Need for Co-Simulation • GECO – Our Co-simulator• Relay Case Study• All PMU-State Estimator

– Experimental Framework– Experimental Results and Interpretations

• Conclusions

Co-Simulation Results

• Use estimated voltage at Bus 3 to represent if the estimation is done successfully

• Attacks at critical locations to show typical vulnerability

Network Link Failure at Bus16-Bus17 (Tp=50ms)

62

Network Link Failure at Bus16-Bus17 (Tp=60ms)

63

Network Link Congestion at Bus16-Bus17

64

Router Congestion: Bus16

65

Data Spoofing: Bus 3

66

Data Spoofing : Bus 3 (with a Real Fault)

67

Outline

• Motivation• Need for Co-Simulation • GECO – Our Co-simulator• Relay Case Study• All PMU-State Estimator

– Experimental Framework– Experimental Results and Interpretations

• Conclusions

Conclusions

• Smart Grid is an ELCPS • Cyber Security Vulnerability for WAMS applications must be studied in

depth• Co-Simulation is a good way to study Smart Grid applications• GECO is built for such studies • These case studies enhanced our confidence in GECO as a tool to study

new smart grid protocols and cyber security impacts on smart grid• Can we draw any general conclusions?

– Possibly not without stretching our imagination– Need for identifying critical bottle neck links and nodes and safe guarding

them– Further studies needed to develop

• More threat models• Defense mechanisms against threat models