Approximate Dynamic Programming and Reinforcement Learning for Nonlinear Optimal Control of Power Systems November 4, 2003 Ronald Harley Georgia Institute.
Post on 20-Dec-2015
221 Views
Preview:
Transcript
Approximate Dynamic Programming and Reinforcement Learning for Nonlinear
Optimal Control of Power Systems
November 4, 2003
Ronald HarleyGeorgia Institute of Technology
ECS-0231632ECS-0080764
Kumar VenayagamoorthyUniversity of Missouri-Rolla
Adaptive Critic Design: Nonlinear Optimal Control
PlantInformaton
UtilityFunction (U)
Optimal cost-to-gofunction (J)
Critic Networks :To minimize the value (of derivatives)
of J with respect to the states
Derivatives via BP
Model Network(Identifier) : To learnthe dynamics of plant
Model Network
Action Network :To find optimal control
u
Plant
Control
Reinforcement Learning
0
* )()(k
k ktUtJ
STATCOM Control
Simulation Results
100ms SC at PCC,
Line Voltage , Generator Terminal VoltageV
The simplified schematic of the SSSC (160 MVA, 15KV VL-L)
Optimal control for FACTS devicesInternal control for static series synchronous
compensator (SSSC)
Series VSI
SynchronousGenerator
V dc
Inf. bus
v s v rv c
is v x
Turbine-Governor
AVR -Exciter
rexe
SSSC
+
GTO
Control
DHPNCCONVC
idc
Optimal control for FACTS devicesInternal control for SSSC (CONVC)
PI Based internal controller (CONVC) for the SSSC
Synchrnouslyrotating reference
transformations
kk ip
+
+
)ˆ
ˆ(tan
ˆˆ
1
22
cd
cq
dc
cqcd
v
vα
V
vvm
Vdc *
Vdc
+-
id
PI- Vdc
iq
- s
kk ip
iq
PI- iq
cqv̂
+iq*
Vdc
s
kk ip
PI- ip
ip
cdv̂+id*
-
ia ib ic
m
+
GTO gate controlof series VSI
+
P*Q*
Vdc
id
Vectorphase-locked
loop
va
vb
vc
Real and reactivecurrent computation
V r
V'dc
Publication: N.G. Hingorani and L. Gyugyi, “Understanding FACTS-Concepts and Technology of Flexible AC Transmission Systems”, IEEE Press, New York, 2000.
Optimal control for FACTS devicesCase study: 100 ms three phase short circuit test at
receiving-end (infinite-bus)
Rotor angle
0 1 2 3 4 5 620
30
40
50
60
70
80
90
100
110
120
130
Time [s]
d [D
egre
e]
Uncompensated
CONVC
DHPNC
Schematic single-line diagram showing an SCRC with external controller (160 MVA, 15KV VL-L)
Optimal control for FACTS devicesExternal control for series capacitive reactance
compensator (SCRC)
SynchronousGenerator Inf. bus
v s
is
SCRC
v r 0
re2xe2
re1xe1Turbine-
Governor
AVR -Exciter
CsT1
1
Internal Controlof SCRC
VoltageSourceInverter
V dc
v c
+GTO
W
WC
sT1
sTK
X C
Filtering DampingController
CsT1
1
External Control
+
+
Line #1
Line #2
*CX
CX
Optimal control for FACTS devicesDHP based external controller (DHPEC)
Schematic single-line diagram showing the DHP based external controller (DHPEC)
SynchronousGenerator
Inf. bus
vs
is
SCRC
re2 xe2
re1 xe1Turbine-Governor
AVR -Exciter
Internal Controlof SCRC
VoltageSourceInverter
Vdc
vc
+GTO
XC
+
+
Line #1
Line #2
*CX
CX
vr
DHP basedexternal controller
(DHPEC)
Optimal control for FACTS devicesCase study: Step changes X*C [pu]
0 5 10 15 20 25-0.1
-0.08
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
Time [s]
[ra
d/s
]
Fixed XC*
KC=1.5
DHPC
Speed deviation
Application in Multi-Machine power system
FACTS(SCRC)
Gen 31600 MVA
Gen 15000 MVA
Gen 22200 MVA 200 km
AREA 1 AREA 2
T1
T2
T3
T4
T5
T6
500 kV 500 kV 500 kV500 kV 13.2 kV
13.2 kV
13.2 kV
13.8 kV
115 kV
Line 1
Line 2
Line 3
Line 4
Line 5
Z 1
Z 2
1
2
34 5
610
7
DHPNC-G
CONVC-GS1
DHPEC-S
CONVEC-SS2 3
11Industrial load
Residentialload
115 kV 13.8 kV
8 9
Large-scale multi-machine power system
A UPFC in the POWER SYSTEM
InfiniteBusShunt
InverterSeries
InverterVd
c
SeriesInverterControl
ShuntInverterControl
V1
Vdcref
R1, L1
V2V1
V1ref
Z1Synch
Generator
Governor
AVR
Exciter
+
-
UPFC
Z1
V1ref
Vdc
Pref
Pinj
Qinj
Q ref
1 2
Pout,
Qout
Verr
Vdcerr Perr
Qerr
R2, L2
Vr
TurbinePref
Neurocontroller
Neuroidentifier
Q
P
ed
eq
Neurocontroller
Neuroidentifier
Vdc
V
epd
epq
5 6 7 8 9 10 11-50
0
50
100
150
200
250
Time (sec)
Load
Ang
le(°
)
UPFC
PI
5 6 7 8 9 10 11-50
0
50
100
150
200
250
Time (sec)
Load
Ang
le(°
)
UPFC
PI
NC
PI
Responses of the Generator for a 180 ms 3- phase Short Circuit at bus 2 at P=0.8 p.u & Q=0.15 p.u
Load angle
Speed response
4 5 6 7 8 9 10 110.96
0.98
1
1.02
1.04
1.06
1.08
1.1
Time (sec)
Spe
ed (
Pu)
UPFC
PI
4 5 6 7 8 9 10 110.96
0.98
1
1.02
1.04
1.06
1.08
1.1
Time (sec)
Spe
ed (
pu)
UPFC
PI
NC
PI
Micro-Machine Research Lab. at the University of Natal, Durban, South Africa
Gen. #1: Trans. Line Impedance Increase
10 10.5 11 11.5 12 12.5 13 13.5 14 14.5 1520
25
30
35
40
Time in seconds
Load
ang
le in
deg
rees
DHP_CONV
CONV_PSS_CONVCON_CONV
10 10.5 11 11.5 12 12.5 13 13.5 14 14.5
0.97
0.98
0.99
1
1.01
Time in seconds
Ter
min
al v
olta
ge in
pu
DHP_CONV
CONV_PSS_CONV
CONV_CONV
top related