This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
I. INTRODUCTION Successful operation of power system depends on the
balance in between total generation and total load demand with associated system losses. Any mismatch between generation and demand causes the deviations of frequency and schedule power exchange among control areas. AGC system plays an important role in electric power system design and operation [1]. Frequency regulation is one of the ancillary services in power system. AGC system consists of secondary frequency control [2]. Secondary frequency control used to restore area frequency to its nominal value. Therefore, AGC used for regains equilibrium between demand and generation by changing unit generation. AGC maintain scheduled system frequency and tie line during normal/abnormal operating conditions [3].
Deregulation is a process for improving efficiency in the operation of power system. In deregulated scenario generation, transmission and distribution works separately and comes under GENCO (generation companies), DISCO (distribution companies) and TRANSCO (transmission companies) respectively. The independent system operator (ISO) is also an important entity of deregulated power system who ensure the secure and reliable operation of the integrated power system for reliable and secure operation of power system ISO procure a number of ancillary services [4]. Researchers have discussed the frequency control issues in deregulated environment [5-8]. In deregulated structure, a DISCO in one area can contract individually with a GENCO of same or different area for power transaction. Donde and Pie introduced the concept of DISCO participation matrix (DPM) for visualization of contract easier on which demand of DISCOs is allocated to
various GENCOs. [9]. A numbers of different approaches such as classical, optimal, genetic algorithm, fuzzy logic, particle swarm, artificial neural network, robust etc. have been used previously to design a controller in AGC schemes [10-12]. An optimal controller for multiarea AGC scheme has been discussed in [13], using the full state feedback. One major drawback with state feedback approach is static feedback gain matrix. In ADHDP approach gain matrix is updated every time and gives optimal performance by minimizing the given objective function. Therefore, in this paper approximate dynamic programming approach has been used to design the optimal controller for multiarea AGC scheme. The discrete model of the system has been utilized to design the controller. First, a quality function has been defined. This function has been used to determine full state feedback gains. The proposed controller has been successfully tested on a two area thermal power system. The results obtained by ADHDP approach has been compared with the discrete state feedback controller.
II. SYSTEM MODELLING In a conventional multiarea (m-area) AGC scheme the
block diagram of ith
area can be represented as shown in fig. 1. Where, in each area, all generators are assumed to be a coherent group. For areas (i,j=1,2,… m) and i≠j, Tgi is the governor time constant, Tpi is the power system time constant, Tti is the turbine time constant, Kpi is the power system gain constant, Ri is the speed regulation, Bi is the frequency bias constant, denotes deviation from the nominal values, fi is the system frequency, Pti is the turbine power, Pgi is the governor output, Tij is the synchronizing coefficient, Ptieij is the tie line power, ACEi is the area control error, Pdi is the load changes.
1Tti.s+1
KpiTpi.s+1
1Tgi.s+1
1/Ri
Bi
-+
-+++ 1
S
Δptiei-j
ui
ΔpdiΔptiei-j
ACEiX2i
X1iX3i
X8i
(a)
Δptiei-j
-+-+
TijS
fj
fi
(b)
Fig. 1. (a) Block diagram for one area of system (ith area). (b) Inter area tie
line power
Based on block diagram shown in fig.1, state space model of two area AGC scheme can be represented as,
( ) ( ) ( )c c cx A x t B u t F d t
Where, state vector ‘ ’ ( , control vector ‘ ( , and
The various matrices, cA (state matrix), cB (control matrix),
and cF (disturbance matrix) are given below:
1
2
10 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0
T
c
TgB
Tg
1
1
2
2
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
T
c
Kp
TpF
Kp
Tp
1 1
1 1 1
1 1
1 1 1
2 2
2 2 2
2 2
2 2 2
0 0
1
2
10 0 0 0 0 0
1 10 0 0 0 0 0 0
1 10 0 0 0 0 0 0
10 0 0 0 0 0
1 10 0 0 0 0 0 0
1 10 0 0 0 0 0 0
2 0 0 2 0 0 0 0 0
0 0 0 0 0 1 0 0
0 0 0 0 0 1 0 0
c
Kp Kp
Tp Tp Tp
Tt Tt
R Tg Tg
Kp Kp
Tp Tp TpA
Tt Tt
R Tg Tg
T T
B
B
This interconnected AGC scheme shown in fig. 1 is
modified to deregulated AGC. In deregulated environment different types of transaction can take place such as bilateral transactions and Poolco based transactions. These transactions can be implemented using the DPM. DPM is a matrix used for visualization of contracts in between GENCO and DISCO [8]. In conventional power system tie line power generally maintained at a fixed value but in deregulated market, a DISCO may have contract with GENCO in same area as well as in other area. Hence ACE signal has to be distributed among them. Coefficients that distribute ACE to several GENCOs are known as “ACE participation factors”. Therefore, the scheduled tie line interchange will change as the demand of DISCO changes. Hence, the net power flow on tie line from an area i can be determined as,
1 1
_m m
i i
j ji j j i
Ptie schd Ptie Dij Dji
Where, Dij is the demand of DISCO in area j from GENCO in
area i, Dji is the demand of DISCO in area i from GENCO in
area j. Therefore, the tie line power error is,
_ _ _i i iPtie error Ptie actual Ptie schd
Therefore, ACE signal changes to,
_ACEi Bi fi Ptiei error
Fig. 2 represents the block diagram of the kth
Genco in area-i. The is the Gencos participation factor, Ri is the droop, and Gg and Gt represents the transfer function model of Governor and turbine respectively, and are expressed as given
in [1], 1
1g
G
GsT
, where GT is the governor time
constant and1
1t
T
GsT
, where TT is the turbine time
constant. 1 2, ,...... ,......g g gk gnP P P P
represents the
change in the output of area-i GENCOs. The net change in area-i generation is
1 2 ... ...gi g g gk gmP P P P P , where m is the
total number of GENCOs in area-i. There may be number of
DISCOs in the ith
area. If 1 2, ,..... ,...L L Lp LkP P P P
represents the change in load demand of DISCOs in the area-i. The net load change in the i
th area is given as
1 2 ... ..L L L Lp LnP P P P P , where n is
the total number of DISCOs in area-i.
Controller
Output
1/Ri
apf
GtGg
Bilateral signals
from same area Bilateral signals
from other area
Δfi
1gP2gP
gmP
giP
Fig. 2. Block Diagram of Genco-k of area-i
The overall block diagram of AGC scheme for an ith
area of m-area power system is shown in fig. 3. The power system block represents the power system dynamics given by
1
pi
pi
K
sT , where
piK is the system gain and is equivalent to
1 iD , where Di is the rate of change of load demand LP to
the change in frequency Δf and is expressed in Hz/pu MW and
piT is the time constant and is equivalent to 2Hi/ (f* Di),
where the parameter Hi is the per–unit inertia constant. In fig.
3, LP is the demand of area i. BTii and BTij are are the
bilateral transaction signals from the DISCOs in the same area-i and other area-j, respectively. In deregulated AGC scheme, each area has two DISCOs and two GENCOs. Therefore, the size of power demand vector increased from 2 to 4. The introduction of uncontracted demand vector comes into picture, if any DISCOs violated the contracts by
demanding more power than the scheduled one. The size of various matrices has been modified to incorporate the transactions. Considering all these changes in conventional AGC, the two area state space model given by (1) is modified in the following manner. The state variables for the considered model of the system are,
1 2 1 2 3 4 1 2 3 4 1 2 1 2GV GV GV GV M M M M tiex w w P P P P P P P P ACE dt ACE dt P
Power demands vector:
1 2 3 4
T
L L L L LP P P P P
Tie-Line
deviatio
n signals
to other
areas
Dij
GENCOPower
System
1/RiBi
DISCO
Tie
Lin
e
Tie
- L
ine
Erro
r
Frequency deviation
signals from other
areas
Bilateral
transaction
signals to Gencos
of other area
BTii
gP
LP
tieiPi
ACEi
BTij
Dji
tiejP
ui1S
Fig. 3. AGC Block Diagram for Area-i
Control inputs: 1 2 3 4
Tu u u u u
Uncontracted demands vector:
1 2 3 4
T
uc uc uc uc ucP P P P P
The state space characterization of the closed loop system in
fig. 3 is,
L UCx Ax Bu FP P
(2)
Where, x is the state vector, u is the control input, LP is the
power demands vector of the DISCOs and ucP is the
uncontracted demand. The state matrix , control matrix and disturbances matrices F has the structure given below,
1 1
1 1
2 2
2 2
1311 12 14
1 1 1 1
2321 22 24
2 2 2 2
31 32 33 34
3 3 3 3
4341 42 44
4 4 4 4
31 41 32 42
0 0
0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
P P
P P
P P
P P
G G G G
G G G G
G G G G
G G G G
K K
T T
K K
T T
cpfcpf cpf cpf
T T T TF
cpfcpf cpf cpf
T T T T
cpf cpf cpf cpf
T T T T
cpfcpf cpf cpf
T T T T
cpf cpf cpf cpf
13 23 14 24
31 41 32 42 13 23 14 24
( ) ( )
( ) ( )
0 0 0 0
cpf cpf cpf cpf
cpf cpf cpf cpf cpf cpf cpf cpf
1 1 1
1 1 1 1
2 2 2
12
2 2 2 2
1 1
2 2
3 3
4 4
1 1 1
2 2 2
10 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0
1 10 0 0 0 0 0 0 0 0 0 0
1 10 0 0 0 0 0 0 0 0 0 0
1 10 0 0 0 0 0 0 0 0 0 0
1 10 0 0 0 0 0 0 0 0 0 0
1 10 0 0 0 0 0 0 0 0 0 0
2
1 10 0 0 0 0 0 0 0 0 0
2
P P P
P P P P
P P P
P P P P
T T
T T
T T
T T
G G
G G
K K K
T T T T
K K Ka
T T T T
T T
T T
T T
T T
AR T T
R T T
3 3 3
4 4 4
1
2
12 12
0
1 10 0 0 0 0 0 0 0 0 0 0
2
1 10 0 0 0 0 0 0 0 0 0 0
2
0 0 0 0 0 0 0 0 0 0 0 12
0 0 0 0 0 0 0 0 0 0 0 12
0 0 0 0 0 0 0 0 0 0 02 2
G G
G G
R T T
R T T
B
B
T T
1 1
1 1
2 2
2 2
33
33
44
44
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0
T
P
G P
P
G P
P
PG
P
PG
apf K
T T
apf K
T TB
Kapf
TT
Kapf
TT
T
The above system has been discretized [14] with sampling
time sT and represented as
1 k kk k k L ucx Ax Bu FP P (3)
Where, xk is the state vector at kth
step, uk is the control
vector at kth
step, and xk+1 is the state vector at k+1th
step. A is
the state matrix, B is the control matrix and F and are the
disturbance matrix for discrete time system.
III. DISCRETE TIME LINEAR QUADRATIC
REGULATOR For the AGC scheme mathematical model described in (3),
if (A, B) is stabilizable, then there exists a feedback gain matrix L such that the closed-loop system is asymptotically stable. Therefore, the objective is to design a control law which is given as,
k ku Lx (4)
Which minimizes the cost function
0
( , ) T T
k k k k k k
k
r x u x Px u Ru
(5)
Where, weighting matrices , and 0.
The value of feedback gain matrix L can be determined by
1( )TTL R B KB B KA (6)
Where, K is the symmetric positive definite matrix which can
be determined by solving discrete Riccati equation, given as 1( ) 0T T T TA KA K A KB B KB R B KA P (7)
By placing the value of ku in (3), the closed loop system is
represented as,
1 ( )k k kx A BL x Gx (8)
Where, G is the closed loop system matrix. Discrete state feedback controller is a static controller i.e., feedback gain matrix is constant. State feedback controller gives good dynamical responses, but determining the optimal control policies for nonlinear systems is difficult to solve analytically. It gives rise in complex state feedback or high order dynamical controller. The main limitation of the static controller is the risk of instability, since the gain matrix is static. ADHDP is an useful technique to design a dynamic feedback controller using approximate dynamic programming. This dynamic feedback controller can overcome the demerits of static feedback controller.
IV. ACTION DEPENDENT HEURISTIC DYNAMIC
PROGRAMMING (ADHDP) ADHDP is a useful technique of Approximate Dynamic
Programming (ADP), for feedback control of systems [14-17]. ADHDP, also known as Q- learning and used to estimate the Quality function, generally known as Q-function or action value function for any policy, optimal or non-optimal. In
ADHDP, first the feedback policy, i.e. kLx is evaluated and
then it is updated by utilizing Q function. Watkins [15] defined the Q function as a sum of the instantaneous cost incurred by taking action from state plus the total cost that would accrue if the fixed policy was followed from the state, mathematically this can be described as,
1( , ) ( , ) ( )k k k k kQ x u r x u V x (9)
Where, 0 1 is the discount factor. ( , )k kr x u is the
instantaneous cost and ( )kV x is the long term cost. For linear
system ( )kV x is quadratic and can be represented as,
( ) ( , ) ( )j k j k T T
k j j j j j i
j k j k
V x r x u x Px u Ru
0
( )i T T
i k i k i k i k
i
x Px u Ru
By placing the value of u and xk+1 from (4) and (8)
respectively, the total cost can be determined as (10), [14-15].
0
( )i T Ti k i k
i
x P L RL x
0
( ) ( ) ( )T i T i T i T
k k k k k
i
V x x D P L RL D x x Kx
(10)
For a linear system with quadratic cost, Q function also
becomes a quadratic function [16-17]. is set to 1 for
simplicity.
1 1 1
0( , ) ( , ) ( )
0
kT T T
k k k k k k k k k
k
xPQ x u r x u V x x u x Kx
uR
( ) ( )kT T T
k k k k k k
k
xx u G Ax Bu K Ax Bu
u
T
kT T
k k Tk
xAx u G K A B
uB
kT T
k k
k
xx u H
u
(11)
Where, 0
0
PG
R
is the block diagonal matrix with
blocks P and R, and ( )T
T
AH G K A B
B
is the
quadratic kernel matrix. Q function can be computed explicitly
for a Linear Quadratic Regulator (LQR) problem which is
quadratic in xk and uk and given as,
( , )xx xu kT T
k k k k
ux uu k
kT T
k k u
k
H H xQ x u x u
H H u
xx u H
u
(12)
Where x u is the column vector concatenation of and
, uH is a symmetric positive definite matrix, and the various
sub matrices are given in (13).
'
'
'
'
xx
ux
xu
uu
H P A KA
H A KB
H B KA
H R B KB
(13)
The sub matrix uuH is symmetric positive definite. By giving
policy and cost function at any time, an improved policy can
be determined as,
1 arg min ( , )k k k
u
L x Q x u (14)
Finally the minimum can be obtained by taking the partial
derivative of ( , )k kQ x u with respect to , setting it to zero.
, 0k k
k
Q x uu
(15)
By placing ( , )k kQ x u value from (12) in (15), control action
can be determined as,
1( )k uu ux ku H H x
0 ux k uu kH x H u (16)
Where, 1
1 ( )k uu uxL H H
is the minimizing policy for
all x. Further a new Q function can be assigned to the policy
and policy improvement procedure can be repeated. In
ADHDP technique, at each iteration gain matrix is updated
according to cost to provide the optimal policy. Therefore
ADHDP provides optimal policy for feedback control by
minimizing the given objective function or total cost.
V. RESULTS AND DISCUSSIONS To check the performance of the ADHDP controller two
area AGC scheme in deregulated scenario is considered. The parameters of AGC scheme are given in table I.
TABLE I. TWO AREA POWER SYSTEM PARAMETERS
( for both area)
governor time constant
s ( for both area)
power system time constant
( for both area)
power system gain constant
s ( for both area) turbine time constant
( for both area) speed regulation
( for both area) frequency bias constant
synchronizing constant
Two cases are considered in this work. In first case load
changes occur in area1 (DISCO1 and DISCO2) only. In first
case load changes occur in the DISCOs of both the areas. The
second case is a contract violation case where DISCO violates
the contract by demanding more power than specified in
contract.
A. Case 1 In this case 0.2 pu load demand change has been
considered in area1 (0.1 pu in DISCO1 and 0.1 pu in DISCO2), and 0.2 pu load demand change in area2 (0.1 pu in DISCO3 and 0.1 pu in DISCO4). All DISOCs have contract with GENCOs for power as per the following DPM.
0.5 0.25 0 0.3
0.2 0.25 0 0
0 0.25 1 0.7
0.3 0.25 0 0
DPM
At t=0, a step increase in each DISCOs of area1and area2 has
been considered. The frequency deviation in area1 and area2
are given in fig. 4(a). The change in the tie line power is
determined by,
1 2,
2 4 4 2
1 3 3 1
tie scheduled
ij j ij j
i j i j
P
cpf PL cpf PL
(17)
1 2, 0.05tie scheduledP pu. The change in the tie line
power is shown in fig. 4(c). Fig. 4(a) shows that frequency
deviation of area1 and area2 settle down to zero at steady
state. Tie line power settles to -0.05 pu, which is the change in
the scheduled power on the tie line as shown in fig. (4c). At
steady state, change in generation of all GENCOs must match
the DISCOs demand. This desired generation of a GENCO in
pu can be expressed in terms of contract participation factors
cpf and the total demand of DISCOs as given as
jMi i Lj
j
P cpf P
. Where, LjP is the total load
demand of DISCO . Therefore total generation will be,
1 1 2 2 3 3 4 4Mi i L i L i L i LP cpf P cpf P cpf P cpf P
For the case under consideration, we have,
1 0.5(0.1) 0.25(0.1) 0 0.3(0.1) 0.105MP pu.
Similarly, 2 0.045MP pu, 3 0.195MP pu, 4 0.055MP
pu, as given in fig. 4(b).
(a)
(b)
(c)
Fig. 4. (a) Frequency Deviations (Hz) (b) Change in Generations (pu MW)
(c) Tie line Power Deviations (pu MW).
It is seen that the net change in generation of area1 and
area2 GENCOs is following the load demand of area1 and
area2 respectively at steady state.
B. Case 2: Contract Violation In this case all transactions are considered same as in case
1. In this case it is considered that DISCO1 draw 0.1 pu more power which is not contracted to any GENCO. This extra power forces the frequency to deviate from nominal value. To simulate this case a total load of 0.3 pu and 0.2 pu have been considered in area1 and area2 respectively. The frequency deviations of area1 and area2 vanish at the steady state as shown in fig. 5(a). Since DPM is the same as in case 2 and the excess load of area1 is taken up by GENCOs of area1, therefore the tie line power shown in fig. 5(c) is the same as in case 2 at steady state. Fig. 5(b) shows that the generation of GENCO3 and GENCO4 is not affected by the excess load of DISCO1. This uncontracted load of DISCO1 is shared by the GENCO1 and GENCO2. To keep the frequency at nominal value, ISO of area1 will supply the excess power from the reserve of GENCO1 and GENCO2.
(a)
(b)
(c)
Fig. 5. (a) Frequency deviations (Hz). (b) Generated power (pu MW).
(c) Tie line power (pu MW).
It is seen that in all cases tie line powers, frequency deviations, and change in generations are stabilized at steady state. Results show that Dynamic responses obtained with the ADHDP controller satisfy the requirements of load frequency control problem. It is evident that dynamic responses obtained with full state feedback controller are more oscillatory and takes more settling time. Results show that both controller works well and settle down the deviations to desired value at steady state but transient responses show that ADHDP controller performance is better than feedback controller. Furthermore the higher flexibility and simple structure of approximate dynamic programming based controller provide solution for a wide range of load disturbances. The performance parameters i.e., overshoot and settling time are given in table II. This table also compares the performance of ADHDP and discrete state feedback controller.
TABLE II.STUDY OF SETTLING TIME AND OVERSHOOT
Controllers Settling Time Overshoot
Δf1 (sec) Δf2 (sec) Δf1
Δf2
Case 1: By ADHDP 120 115 -0.42 -0.43
By feedback controller 135 125 -0.25 -0.26
Case 2 : By ADHDP 200 200 -0.72 -0.23
By feedback controller 300 300 -0.66 -0.25
VI. CONCLUSION In this paper, Action Dependent Heuristic Dynamic
Programming (ADHDP) based controller has been designed for a two area automatic generation control scheme in deregulated scenario. Bilateral market transactions have been considered in between GENCOs and DISCOs. Different load conditions have been considered to test the controller. Results of ADHDP based controller has been compared with state feedback controller. Result shows that the ADHDP based controller performs satisfactory for any load perturbation. The results indicate that ADHDP control method provide the control system that satisfy the Automatic Generation Control requirements. The above approach can be extended to multi area power system considering the various non linearities.
REFERNCES [1] I. Egido, FF. Bernal, L. Rouco., “The Spanish AGC system: description
and analysis,” IEEE Trans Power Syst, vol. 24, pp. 271–8, 2009. [2] IP. Kumar, DP. Kothari., “Recent philosophies of automatic generation
control strategies in power systems,” IEEE Trans Power Syst, vol. 20,
pp. 346–57, 2005. [3] J. Nanda, S. Mishra, LC. Saikia., “Maiden application of bacterial
foraging-based optimization technique in multiarea automatic generation
control,” IEEE Trans Power Syst, vol. 24, pp. 602–9, 2009. [4] AM. Pirbazari, “Ancillary services definitions, markets and practices in
the world”, IEEE/PES Transmission and Distribution Conference and
Exposition: Latin America, 2010. [5] J. Kumar, KH. Ng, G. Sheble, “AGC simulator for price based operation
part I” , IEEE Trans. Power Systems, vol. 12, No. 2, pp. 527-532, May 1997.
[6] RD. Christie, A. Bose, “Load frequency control issues in power system
operation after deregulation,” IEEE Transactions on Power Systems, vol. 11, No.3, pp. 1191-1200,1996.
[7] B.Tyagi, S.C. Srivastava, “Automatic generation control scheme based
on dynamic participation of generators for competitive electricity
markets,” in Proc. National Power Systems (NPSC), December 16-18,
IIT-Bombay, 2008.
[8] CS. Chang, W. Fu, “Area load frequency control using fuzzy gain scheduling of PI controllers,” Electric Power Systems Research, vol. 42,
pp.145-/152, 1997.
[9] V. Donde, MA. Pai, IA. Hiskens, “Simulation and optimization in an AGC system after deregulation,” IEEE Transactions on Power System,
vol.16, No. 3, pp. 481-489,2001.
[10] SP. Ghoshal, SK. Goswami, “Application of GA based optimal integral gains in fuzzy based active power-frequency control of non-reheat and
[17] T Landelius, “Reinforcement Learning and Distributed Local Model Synthesis,” Dissertation, Department of Electrical Engineering Linkoping University, Sweden, 1997.