Pinpointing the security boundary in high-dimensional spaces using importance sampling Simon Tindemans, Ioannis Konstantelos, Goran Strbac Risk and Reliability.

Pinpointing the security boundary in high-dimensional spaces using importance sampling

Simon Tindemans, Ioannis Konstantelos, Goran Strbac

Risk and Reliability Modelling of Energy Systems12th November 2014, Durham University

Off-line support for operational security

Day ahead and real time operation

operating conditions

forecasts

contingencies

Security analysis actions

severe computational constraints

1. Anticipate 2. Analyse 3. Classify

Offline analysis (e.g. week ahead)

Monte Carlosampling ofoperatingconditions

contingencies

dynamic simulation

impact analysis

machine learning

data driven heuristics

High-dimensional DT with training errors

Decision trees for security studies

insecure

insecure secure*

secure

parameter 1

para

met

er 2

Two-dimensional example

DT image courtesy of Pepite

The security boundary

parameter 1

para

met

er 2

Quality of predictive classification (secure / insecure)

Pinpointing the security boundary

Which one?

Scenarios ‘near’ the security boundary improve prediction quality

Importance sampling 1. Which states to sample2. How to sample those states

Importance sampling for classification

Previous applications have relied on three assumptions:

1. Meaningful definition of ‘distance’ from the security boundary.

2. ‘Easy’ sampling distributions.

3. ‘Nice’ properties of the security boundary.

We propose a data-driven iterative importance sampling method that does not rely on these assumptions.

Krishnan et al. (2011), IEEE Transactions on Power Systems

Lund et al. (2014), IEEE Transactions on Power Systems

What to sample?

insecure

insecure secure*

secure

parameter 1

para

met

er 2

High-dimensional DT with training errorsTwo-dimensional example

DT image courtesy of Pepite

Defining ‘interestingness’

How to sample?

Repurpose machine learning process to guide samplingDecision trees express security and interestingness in terms of pre-fault variables

Abort evaluation of uninteresting pointsPossible because of separation of time scales

contingencies

dynamic simulation

impact analysis

machine learning

sample random

‘external’ conditions

complete starting

point

~10ms* ~1 min N x ~1 min

Decision Trees

importance sampling

filter

reject manyuninteresting points

Considerations and challenges

How to control biasing?• Biasing parameter b which controls relative populations. • b=0.5 is a defensive choice (max 2x slowdown).• For very high rejection rates, initial stages are no longer negligible.

Weights

Weights should be used at every subsequent analysis step.

Two-stage filteringFurther gains can be made by exploiting gap in effort between sampling of ‘TSO-external’ variables (~ms) and completion of base state (~1 min)

Case study

• Dynamic simulation study based on French EHV grid• ~1500 nodes, ~2000 lines• 30,334 classifying variables• 1970 contingencies• 6 security indices (only overloads used)

• Computation on PRACE Curie HPC• 10,000 cores, 24 hours [ ~ 2.5 tCo2]• 2GB results file; 10GB decision trees• Unbiased sample of 10,044 valid initial conditions

PRACE Curie : http://www-hpc.cea.fr/en/complexe/tgcc-curie.htm

Case study [contd]

‘Offline’ simulation of importance sampling• Use 6,000 states x 1,970 contingencies as an unbiased sample ‘pool’• Process in batches

• generate trees after each batch

Importance sampling acceptance rate• average: 24% (1431 of 6000)• minimum: 16% (967) [least interesting]• maximum: 100% (6000) [most interesting]

Validation using 4,044 states x 1,970 contingencies to estimate errors• Importance sampling classifiers• Unbiased classifiers, using identical computational budget

(1431 states/contingency)

500 500 1000 1000 1000 1000

Results: error analysis

mean change in error : -0.0012

mean number (1431)

misclassification error

poin

ts a

naly

sed

increased attention on badly classified contingencies

computational budget for naïve implementation

decreased average error

without ISwith IS

per contingency:

Results: error analysis - continued

|dError| > 0.01 only : 101 of 1970 contingencies

misclassification error

poin

ts a

naly

sed

Focused analysis results in reduction of errors

Some trees are worse off

mean change in error : -0.016Most change is for the better

Summary and outlook

Summary• Offline analysis and machine learning can support power system

operation• Challenge to pinpoint security boundary with finite resources• Proposed data-driven importance sampling method that uses

‘interestingness trees’ and accept-reject sampling• Initial trials suggest increase in accuracy for given computational

budget

Outlook• Quantification of speedup• Two-stage importance sampling (extra early rejection step) • Implementation on HPC platform

Thank you

This research was supported by the iTesla project within the 7th European Community Framework Programme

Partners for this work

Two-stage importance sampling

contingencies

dynamic simulation and impact

machine learning

sample random

‘external’ conditions

complete starting

point

~10ms* ~1 min

Decision Trees

stage II importance sampling

reject manyuninteresting points

stage I importance sampling

reject manyuninteresting points

reduced classification

N x ~1 min

Example decision tree

Decision tree for classification 1 if MTAHUP6_S_VL6_PGEN<-0.153883 then node 2 else node 3 2 if ROMAIP6_S_VL6_QSHUNT<54.9982 then node 4 else node 5 3 class = false 4 if TAMAR6COND_11_SC_V<244.195 then node 6 else node 7 5 if BXLIEL61ZGRA6_ACLS__TO__ZGRA6P6_S_VL6_V<242.056 then node 8 else node 9 6 if ANSERL61PRRTT_ACLS__TO__ANSERP6_S_VL6_Q<-47.9415 then node 10 else node 11 7 class = false 8 class = true 9 class = false10 class = false11 if BOCTOL71N_SE1_ACLS__TO__N_SE1P7_S_VL7_V<408.882 then node 12 else node 1312 class = false13 class = true

Importance sampling

Importance sampling deliberately distorts the sampling of system states to focus on the “important” events (i.e. those that contribute to the risk metrics).

Simulation results are corrected for this bias by sample weights. If done correctly, this procedure leads to large speed-ups.

𝐸𝑓ሾ𝑞ሺ𝑥ሻሿ= න𝑞ሺ𝑥ሻ𝑓ሺ𝑥ሻ𝑑𝑥= න𝑞ሺ𝑥ሻ𝑓ሺ𝑥ሻ𝑔ሺ𝑥ሻ𝑔(𝑥)𝑑𝑥= 𝐸𝑔ቈ𝑞ሺ𝑥ሻ𝑓ሺ𝑥ሻ𝑔ሺ𝑥ሻ