UNIVERSIDAD DE CASTILLA-LA MANCHA DEPARTAMENTO DE INGENIER ´ IA EL ´ ECTRICA, ELECTR ´ ONICA, AUTOM ´ ATICA Y COMUNICACIONES STOCHASTIC BILEVEL GAMES APPLICATIONS IN ELECTRICITY MARKETS TESIS DOCTORAL AUTOR: DAVID POZO C ´ AMARA DIRECTOR: JAVIER CONTRERAS SANZ Ciudad Real, Diciembre de 2012
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
UNIVERSIDAD DE CASTILLA-LA MANCHA
DEPARTAMENTO DE INGENIERIA ELECTRICA,
ELECTRONICA, AUTOMATICA Y COMUNICACIONES
STOCHASTIC BILEVEL GAMES
APPLICATIONS IN ELECTRICITY
MARKETS
TESIS DOCTORAL
AUTOR: DAVID POZO CAMARA
DIRECTOR: JAVIER CONTRERAS SANZ
Ciudad Real, Diciembre de 2012
UNIVERSIDAD DE CASTILLA-LA MANCHA
DEPARTMENT OF ELECTRICAL ENGINEERING
STOCHASTIC BILEVEL GAMES
APPLICATIONS IN ELECTRICITY
MARKETS
PhD THESIS
AUTHOR: DAVID POZO CAMARA
SUPERVISOR: JAVIER CONTRERAS SANZ
Ciudad Real, December 2012
A mi madre.
Por su extraordinaria fuerza y coraje.
Preface
This thesis addresses the subject of bilevel games and their application for
modeling operational and planning problems in restructured power systems.
Such games are well fitted to model hierarchical competition but they are hard
to solve in general. Bilevel games set new challenges for power system operators
and planners and they constitute an ongoing topic for many researchers.
Bilevel games are generally modeled as equilibrium programs with equilib-
rium constraints (EPEC) within the operations research field. EPEC problems
are highly non-linear and non-convex, and the existence of global and unique
solutions is not guaranteed even in the simplest instances of EPECs. Hence, a
generalized theory and solution algorithms for solving EPECs have not been
firmly established so far. Only a few and specific instances of EPECs have been
shown to have equilibria. In many of these instances, the solution is stated as a
stationary equilibrium, which is not necessarily a global solution. Additionally,
most of the proposed solution techniques do not guarantee finding all pure
Nash equilibria. The difficulties both from a theoretical and a numerical point
of view arise because EPEC problems inherit the bad properties of the set of
MPEC problems that conform the corresponding EPEC.
In this thesis, we propose a special case of EPECs where leaders compete
among themselves at the upper level in a Nash equilibrium setting by making
decisions in finite strategies constrained by the solution of the lower level
problem, where the followers compete among themselves in a Nash equilibrium
setting by making continuous decisions. The upper and lower level problems
are linear and uncertainty is included at the lower level. Then, the bilevel
game is stated as a finite stochastic EPEC with the possibility of multiple
equilibria. This specific EPEC structure is appropriate for many problems
that appear in restructured power systems. We devote two chapters of this
thesis to show the applicability of this game structure in both operational and
planning frameworks.
ii
To overcome the difficulties described above, we propose a mixed integer lin-
ear reformulation (convexification) of the corresponding stochastic finite EPEC
problem. The advantage of this approach is two-fold. First, the linearized
formulation can be solved with standard mixed integer linear programming
(MILP) solvers and a global solution can be guaranteed for moderately-sized
problems. Second, the discrete strategies at the upper level problem allow us
to find all (pure) Nash equilibria. This is done by including a set of linear
constraints in the problem that represent “holes” in the feasible region for the
known Nash equilibria.
Finally, although the proposed methodology has several advantages, it is
important to recall its limitations. First, the linearization (convexification)
approach proposed in this thesis requires the inclusion of binary variables into
the model, which increases its complexity. And second, the lower-level problem
has to be a convex optimization problem (linear in this thesis) in order to
transform it into its equivalent and sufficient first-order optimality conditions.
Each chapter is fairly independent but they all share the same mathematical
notation. In Chapter 1 we give an overview of restructured power systems
and a review of the existing literature related with this thesis. In Chapter 2
we describe the mathematical framework for solving stochastic EPECs with
finite strategies. We apply the proposed stochastic EPEC models to electricity
markets in Chapters 3 and 4 in an operational and a planning framework,
respectively. In Chapter 3, a strategic bidding problem is proposed, where
electricity producers compete in the spot market. In Chapter 4 we present a
three-level problem for transmission and generation expansion. To conclude
the thesis, a short summary, conclusions and some hints on future research
topics are given in Chapter 5.
iii
Acknowledgments
This thesis would not have been possible without the financial support of
several institutions and the advice and guidance of many people.
I would like to express my deepest gratitude to my supervisor, Professor
Javier Contreras, for his excellent supervision, dedication, guidance and sup-
port over the past few years.
I am indebted to several relevant people that have helped with their sug-
gestions to add significant value to this thesis. They are not only relevant for
their suggestions, but also for their hospitality and for the exceptional human
and intellectual environment created. First of all, I wish to thank Professor
Felix F. Wu for giving me the opportunity to spend three months in 2009 and
six months in 2010 with his research group at the University of Hong Kong. I
would like to thank to Dr. Yunhe Hou for giving me the opportunity to visit
him at the University of Hong Kong for one month in 2011. I am also obliged
to Dr. Huifu Xu for receiving me in his research group at the University of
Southampton, United Kingdom, for three months in 2011. I would like to
acknowledge Antonio Canoyra, Antonio Guijarro and Angel Caballero, from
Gas Natural Fenosa company, for their suggestions at the beginning of this
thesis and their fruitful feedback to apply the models developed to the real
world. I am also obliged to Dr. Jose Ignacio Munoz and Dr. Javier Dıaz. My
sincere thanks to Professor Enzo E. Sauma for his relevant suggestions in this
work. It is also worth mentioning the contribution of Professor Sauma as a
co-author of two papers related to this thesis.
I thank several institutions that have supported my PhD studies allowing
me to spend part of this time at the University of Hong Kong, and at the
University of Southampton. First of all, I would like to thank Gas Natural
Fenosa company for their financial support at the beginning of my PhD. Also,
I wish to thank Junta de Comunidades de Castilla-La Mancha of Spain for its
financial support through the program “Formacion del Personal Investigador”
iv
grant 402/09. Additionally, I thank the University of Hong Kong for their
support during my visit. I am also indebted to the Universidad de Castilla-La
Mancha for allowing me to use its facilities and the financial support from the
program “Ayudas a la Investigacion para la realizacion de Tesis Doctorales”.
I wish to acknowledge all my colleagues and good friends I have made during
these years at the Escuela Tecnica Superior de Ingenieros Industriales at Ciu-
dad Real, at the Electrical Engineering Department of the University of Hong
Kong and at the School of Mathematics at the University of Southampton.
Claudia, Virginia, Agustın, Alberto, Rafa, Alex Street, Jesus Lopez, Cristiane,
Luis, Valentın, Juanda, Carlos Rocha, Roberto Lotero, Wilian, Rafaella, Diego,
The objective function for the i-th leader consists of the minimization of
the expected payoff function, Fi, that depends on their own strategies, xi, the
30 2. Mathematical Framework for Bilevel Games
strategies their competitors, x−i, the followers’ decisions, y, and the Lagrange
multipliers of the lower-level problem, λ and µ. Leader i chooses an optimal
value, xi ∈ Xi ⊆ ZKi . The upper-level decisions are finite decisions that help
to overcome the difficulties for finding global solutions.
S(xi,xe−i, ξ) denotes the solution of the follower’s equilibrium problem pa-
rameterized by the leader’s strategies. The constraint (y(i),λ(i),µ(i)) ∈ S(xi,xe−i, ξ)
is an equilibrium constraint and it can be replaced by variational inequalities
[80], complementary constraints [81], or an optimization problem resulting in
a bilevel program [66].
The lower-level variables y(i), λ(i) and µ(i), are parameterized in terms of
the i-th leader’s decision, xi, with the competitors fixed at the equilibrium
decision, xe−i, i.e. y(i) = y(i)(xi,xe−i, ξ). Note that, although the followers
are the same for all leaders, the followers’ optimal responses (and Lagrange
multipliers) can be different for each leader.
In our problem, the leaders’ decisions are made under uncertainty at the
upper level. The followers’ decisions are made with full knowledge of the
stochastic variables at the lower level. This problem has similarities with a
two-stage stochastic optimization problem, where some decisions are made at
the first stage before knowing the scenario realization, and other decisions
are made after the scenario realization is known. In this sense, a set of m
optimization problems at the lower level is solved for each scenario and each
leader.
The followers’ equilibrium problem parameterized by the decisions of the
i-th leader can be stated as a set of optimization problems for each follower
j. Then, (y(i), λ(i), µ(i)) ∈ S(xi,x
e−i, ξ) is a solution to the lower-level Nash
equilibrium if and only if:
(y(i), λ(i), µ(i)) solves,
∀j = 1, . . . ,m
minimizey(i)j
fj
(xi,x
e−i, y
(i)j , y
(i)−j, ξ
)subject to: y
(i)j ∈ Yj(y(i)
−j) ⊆ RKj
+
(2.2)
Follower j chooses their optimal decision minimizing their payoff function,
fi, conjecturing the reactions of their competitors, y(i)−j, at the equilibrium.
2.1. Introduction 31
Note that the set of m problems for all the followers is parameterized by the
leader’s decisions, xi and xe−i.
The proposed EPEC problem (2.1)–(2.2) is highly non-linear and non-
convex, therefore, existence and uniqueness of equilibrium points rarely hap-
pens. Global solutions are seldom reached for the algorithms proposed in
literature. Therefore alternative local solutions are drawn for solving EPECs
as local Nash equilibrium or Nash stationary equilibrium. Specific equilibrium
definitions depend on the constraint qualification of the problem, such as the
W- (Weakly), C- (Clarke), B- (Bouligand), M- (Mordukhovich) or S-stationary
(Strongly stationary) equilibrium. Reference [86] defines such equilibria for
solving MPCCs.
Finding algorithms to solve EPECs constitutes an ongoing line of research.
The two main algorithms suggested for solving EPECs are based on a di-
agonalization approach or the simultaneous solution of the strong stationary
necessary conditions for all individual MPECs. But only in few special cases
global solutions are reached with these algorithms. Furthermore, multiple equi-
libria solutions are possible, but the algorithms for finding all Nash equilibria
are not implemented.
We have solved all these problems by converting the stochastic EPEC into
an MILP optimization problem. Consequently, global optimality is guaranteed
at the expense of the tractability of the problem. Indeed, the algorithm is
limited for solving problems efficiently where the lower-level problem is linear.
Our algorithm has two special features: i) finite decisions are made at
the upper-level, and ii) a combinatorial approach is used for transforming the
SEPEC into an MILP. In regards to the former, this is not always a problem,
because some problems require finite decisions, as in the transmission and
generation capacity investment problem or the annual generator maintenance
[91]. Nevertheless, other problems modeled with continuous decision variables
would require to discretize them in order to apply our approach. A fine
discretization is closer to the continuous variable case but it involves a higher
number of variables and constraints and the problem may not be tractable.
Regarding the second feature, the problem is limited to solve easy instances of
32 2. Mathematical Framework for Bilevel Games
the lower-level problem, such as linear and quadratic lower-level optimization
problems. In this thesis we deal with bilevel linear models where the upper-
level and lower-level problem are cast as LPs.
2.2 Game Theory Definitions
A game is a formal representation of a situation in which a number of players
interact in a setting of strategic interdependence [11]. This means that the
welfare of a player depends upon their own action and the actions of the
other players in the game. A game can be either cooperative, where the
players collaborate to achieve a common goal, or noncooperative, where they
act for their own benefit. Also, a game can be either of perfect or imperfect
information, and sequential or simultaneous (the players play at the same
time).
A player plays a game through actions. An action is a choice or election
that a player makes, according to their own strategy. A strategy is a rule
that tells the player which action(s) they should take, according to their own
information set at any particular stage of a game. Finally, a payoff function
expresses the utility that a player obtains given the strategy profile of all the
players.
Assume that there is a finite set of players, i = 1, . . . , n participating in
a game. Each player can take an individual strategy represented by a vector
xi. The overall strategies taken by all players are represented with the tuple
x = (x1, . . . , xn). The rivals’ strategies are represented by the tuple x−i =
(x1, . . . , xi−1, xi+1, . . . , xn) that denotes all the players strategies except for
player i. Xi denotes the strategy space of player i. Xi can be either continuous
or integer, a convex or non convex set where the strategies can take place. For
example Xi can be defined as the set Xi = xi ∈ RKi : hi(xi) = 0, gi(xi) ≤ 0,where Ki is the number of variables, xi, controlled by player i, i.e., it is the
size of vector xi.
By ui(xi,x−i) : X1×X2, . . . , Xi, . . . , Xn 7→ R we define the payoff function
for player i. In this chapter, the payoff function is considered as a cost function
or a minus profit function. Therefore, the players are interested in minimizing
2.3. One-Level Games 33
their payoff functions.
2.3 One-Level Games
2.3.1 Nash Equilibrium Problem
Amongst all the definitions of equilibria, the Nash equilibrium is the most
widely used. The pure Nash equilibrium constitutes a profile of strategies such
that each player’s strategy is the best response to the other players’ strategies
that are actually played. Therefore, no player has an incentive for changing
their strategy. More formally, a strategy vector xe = (xe1, . . . , xen) is the pure
Nash equilibrium of a game if (2.3) is satisfied for all players.
ui(xei ,x
e−i) ≤ ui(xi,x
e−i), ∀xi ∈ Xi, ∀i = 1, . . . , n (2.3)
Note that xe solves the game in the following sense: at xe no player can
improve their individual payoff unilaterally. In essence, each player faces an
optimization problem measured by their payoff function. The set of coupled
optimization problems represents a Nash equilibrium problem (NEP). Another
equivalent definition of equation (2.3) for the (pure) Nash equilibrium is given
by (2.4), where the NEP is stated as a set of coupled optimization problems.
xei solves,
∀i = 1, . . . , n
minimizexi
ui(xi,xe−i)
subject to: xi ∈ Xi
(2.4)
The NEP has been widely studied, and conclusions about its existence
and uniqueness have been drawn. In his first definition [10], Nash proved the
existence of the solutions through the Kakutani’s fixed point theorem when
the payoff functions for each player are assumed to be concave for each xi.
34 2. Mathematical Framework for Bilevel Games
2.3.2 Generalized Nash Equilibrium Problem
If the actions available for the players depend on the decisions made by their
rivals (i.e. Xi = Xi(x−i)) the game is known as generalized Nash equilibrium
problem (GNEP). This term was introduced by Harker [106]. The GNEP has
a wide range of applications but it is more difficult to solve than the standard
NEP.
Equations (2.5) and (2.6) represent the GNEP as a system of inequalities
or as a set of optimization problems, respectively.
ui(xei ,x
e−i) ≤ ui(xi,x
e−i), ∀xi ∈ Xi(x
e−i), ∀i = 1, . . . , n (2.5)
xei solves,
∀i = 1, . . . , n
minimizexi
ui(xi,xe−i)
subject to: xi ∈ Xi(xe−i)
(2.6)
In the next example we give a graphic interpretation for the NEP and
GNEP strategy spaces for a two-player game.
Example 2.1 Given a two-player game, player 1 chooses amongst the strate-
gies x1 ∈ X1 ⊆ R and player 2 chooses amongst the strategies x2 ∈ X2 ⊆ R,
given the payoff functions, u1(x1, x2) : X1×X2 7→ R for player 1 and u2(x1, x2) :
X1 ×X2 7→ R for player 2.
The NEP for the two-player game is defined as (2.7).
(xe1, xe2) solves
minimizex1
u1(x1, xe2), s.t. x1 ∈ X1 ⊆ R
minimizex2
u2(xe1, x2), s.t. x2 ∈ X2 ⊆ R
(2.7)
In the GNEP two-player game, the set of strategies of player 1 depends
on the decisions of player 2. So player 1 can choose among the strategies
2.3. One-Level Games 35
x1 ∈ X1(x2) ⊆ R, where X1(x2) represents the parameterized domain set of x1
in terms of their competitor, x2. In the same vein, player 2 chooses among the
strategies x2 ∈ X2(x1) ⊆ R. Therefore, at the (pure) Nash equilibrium point,
(xe1, xe2), the domain sets of strategies are defined as x1 ∈ X1(xe2) ⊆ R for player
1 and x2 ∈ X2(xe1) ⊆ R for player 2.
The GNEP for the two-player game is defined as a set of optimization
problems (2.8).
(xe1, xe2) solves
minimizex1
u1(x1, xe2), s.t. x1 ∈ X1(xe2) ⊆ R
minimizex2
u2(xe1, x2), s.t. x2 ∈ X2(xe1) ⊆ R
(2.8)
Figure 2.2 shows an example of the (closed and convex) space of the strate-
gies sets for the two-player game in the case of solving the NEP (left hand
side) or solving the GNEP (right hand side).
x2
x1
X1
X2
x2
x1
X1
X2
X2(x1)
X1(x2)
Figure 2.2: Example of (closed and convex) sets of strategies: Left for theNEP defined in (2.7); Right for the GNEP defined in (2.8)
Notice that a pure Nash equilibrium must always belong to the intersection
of the overall players’ strategic spaces. Therefore, the two-player equilibrium
must belong to the set X(x1, x2) ⊆ R2 = X1 ∩X2 for the NEP or X(x1, x2) ⊆R2 = X1(x2)∩X2(x1) for the GNEP. This motivates the next Nash equilibrium
definition.
36 2. Mathematical Framework for Bilevel Games
2.3.3 Generalized Nash Equilibrium Problem with Shared
Constraints
A GNEP with shared constraints is a special instance of GNEP with coupled
constraints (see equations (2.5) and (2.6)). In this game there exists a set
of common constraints that simultaneously restrict each player’s optimization
problem. Shared constraints games were introduced by Rosen in 1965 [107],
who proved the existence and uniqueness of the equilibrium when the set of
shared constraints is closed, convex and bounded and the payoff functions
satisfy diagonal strict concavity. In a recent paper [108] and a PhD dissertation
[109], Kulkarni claimed finding the global pure Nash equilibrium for bilevel
games with shared constraints and potential payoff functions [110].
Because the GNEP (with coupled constraints) is almost intractable, some
authors propose to convert the original problem into a GNEP with shared
constraints [36, 108]. The modifications consist of including the competitors’
constraints set for each player. This is equivalent to add the overall player’s
set of space constraints, X(x), to each optimization problem, that is defined as
the intersection of all the players’ strategies spaces, i.e. X(x) =⋂ni=1 Xi(x−i).
The GNEP with shared constraints is defined as a set of inequalities (2.9)
or a set of optimization problems (2.10).
ui(xei ,x
e−i) ≤ ui(xi,x
e−i), ∀xi ∈ X(xi,x
e−i), ∀i = 1, . . . , n (2.9)
xei solves,
∀i = 1, . . . , n
minimizexi
ui(xi,xe−i)
subject to: xi ∈ X(xi,xe−i)
(2.10)
We illustrate the GNEP for coupled constraints and shared constraints for
the two-player game in Example 2.2.
Example 2.2 Based on Example 2.1 for a two-player game, the equivalent
GNEP with shared constraints is defined as:
2.3. One-Level Games 37
(xe1, xe2) solves
minimizex1
u1(x1, xe2), s.t. x1 ∈ X(x1, x
e2) ⊆ R
minimizex2
u2(xe1, x2), s.t. x2 ∈ X(xe1, x2) ⊆ R
(2.11)
where X(x1, x2) = X1(x1, x2) ∩X2(x1, x2).
Figure 2.3 illustrates the strategy spaces for the GNEP at the left hand side
and for the GNEP with shared constraints at the right hand side. For both
problems a Nash equilibrium solution must hold in the X(x1, x2) space. But,
as can be seen in Figure 2.3, any player strategy space is more restricted for the
shared constraints case than for the general one. For example, when player 1
chooses xe1, player 2 optimizes their payoff function over X2(xe1) ⊆ R. This set
is more constrained for the shared constraints case than for the coupled one.
Because the space of strategies changes for both problems, the Nash equilib-
ria may differ between both game representations.
X2(x1)
X1(x2)
(xe1, x
e2)
x2
x1
X1(xe2)
X2(xe1)
x2
x1
(xe1, x
e2)
X1(xe2)
X2(xe1)
X(x1, x2)
Figure 2.3: Example of (closed and convex) sets of strategies: Left for theGNEP with coupled constraints defined in (2.8); Right for the GNEP withshared constraints defined in (2.11)
Due to the modification of the strategy space for the players in the shared
constrained case, the solutions of both problems may differ. A solution of the
GNEP with coupled constraints problem is a solution of the GNEP with shared
38 2. Mathematical Framework for Bilevel Games
constraints, but not viceversa (see [109]). Therefore the modified GNEP with
shared constraints has at least the same Nash equilibria as the GNEP with
coupled constraints. For further details about GNEP with shared constraints
see [36, 108,109].
In the next example we illustrate the solution set obtained for the NEP,
GNEP, and GNEP with shared constraints.
Example 2.3 Based on the previous two-player game from Examples 2.1 and
2.2 we define a linear payoff function for both players. The gradients of their
objective functions are ∇u1(x1, x2) for player 1 and ∇u2(x1, x2) for player 2.
They are represented in Figures 2.4, 2.5 and 2.6 with the space of strategies
for each player. The arrows point at the optimization direction of the objective
function for each player.
The NEP solution is illustrated in Figure 2.4. There is a single Nash
equilibrium located in one vertex of the space of strategies. Note that for any
other point of the strategies set, player 1 always chooses the highest value of x1,
given any competitor’s strategy. Similarly, player 2 always chooses the highest
value of x2, given any x1. From the space of strategies it is easy to deduce that
there is only one Nash equilibrium.
x2
x1
X1
X2
∇u1(x1, x2)∇u2(x1, x2)
NE
Figure 2.4: NEP solution from equation (2.7)
The GNEP is illustrated with Figure 2.5, where is also a single generalized
Nash equilibrium. Notice that for a fixed strategy of player 2, xe2, player 1
2.3. One-Level Games 39
chooses the highest value of x1 ∈ X1(xe2). And for player 1 fixed at xe1, player 2
chooses the highest value x2 ∈ X2(xe1). There is only a single point where both
players minimize their payoff functions simultaneously and they do not have
better alternatives to choose. It is the GNE shown in Figure 2.5.
x2
x1
∇u1(x1, x2)∇u2(x1, x2)
GNE
X2(x1)
X1(x2)
X2(xe1)
X1(xe2)
Figure 2.5: GNEP solution from equation (2.8)
The GNEP with shared constraints is illustrated in Figure 2.6. If player 1
is fixed at any point of the set of GNE xe1, player 2 chooses x2 ∈ X2(xe1) =
X(xe1, x2), and the point in the thick boundary is the one that minimizes the
payoff function for player 2. Analogously, player 1 does not deviate from any
fixed point of player 2 placed in the thick red line. Therefore, the thick red line
represents an infinite number of GNEs. GNEs have different objective values
for both players. The infinite number of GNEs includes the equilibrium for the
general case where the constraints are not shared.
A stochastic GNEP is an extension of the GNEP including uncertainty. Among
several possible formulations we provide one in which the payoff function is
based on the expected values and solved as a stochastic optimization problem
[111].
40 2. Mathematical Framework for Bilevel Games
∇u1(x1, x2)∇u2(x1, x2)
Set of GNE solutions
x2
x1
X(x1, x2)
X2(xe1)
X1(xe2)
Figure 2.6: GNEP with shared constraints solutions from equation (2.11)
Some stochastic optimization problems include risk measures for hedging
against uncertainty. But, in general, those problems have many Pareto-efficient
solutions. Different attitudes about risk imply different costs (or profits). Such
risk attitudes are selected by the decision maker in terms of risk aversions.
Because a risk attitude is not always evident for the decision maker and,
therefore, for their competitors, the Nash equilibrium problem including risk
hedging has a difficult economic interpretation. Some approaches for solving
stochastic Nash equilibria as robust NE problems (or worst-case) are studied
in [112] and [113] in terms of the expected values. Some authors [102] have
included risk defined as Conditional Value at Risk (CVaR) [114] in the payoff
function as a penalty term for each player, but risk aversion is assumed equal
for all players and is chosen arbitrarily.
Considering risk neutral players, the stochastic GNEP is given by:
xei solves,
∀i = 1, . . . , n
minimizexi
E[ui(xi,x
e−i, ξ)
]subject to: xi ∈ Xi(x
e−i, ξ)
(2.12)
The stochastic GNEP involves some random variables represented by ξ.
A sample average method is used for solving stochastic problems because
they have two specific features: the random variable is seldom fully known
2.3. One-Level Games 41
and, even if it is known, solving the problem with this function makes it
non tractable. Therefore, a sampling method of scenarios, like Monte Carlo
simulation, resolves these problems and the stochastic optimization problem
becomes an equivalent deterministic optimization one. Equation (2.13) shows
the scenario-based optimization problem formulation.
xei solves,
∀i = 1, . . . , n
minimizexi
E[ui(xi,x
e−i, ξ(ω))
]subject to: xi ∈ Xi(x
e−i, ξ(ω))
(2.13)
2.3.5 Finite-Strategy Nash Equilibrium Problem
Finite-strategy games or just finite games have been widely studied in lit-
erature since J. F. Nash formulated the equilibrium problem in [10] with
finite decisions. In these games, the players have a finite set of strategies.
Therefore, the set of overall actions that the i-th player can select is xi ∈ Xi =
x1i , x
2i , . . . , x
Kii , where Ki is the total number of strategies that player i can
choose.
Based on the previous definition of the NEP, the finite NEP is formulated
as a set of inequalities (2.14) or as a set of optimization problems (2.15).
ui(xei ,x
e−i) ≤ ui(xi,x
e−i), ∀xi ∈ x1
i , . . . , xKii , ∀i = 1, . . . , n (2.14)
xei solves,
∀i = 1, . . . , n
minimizexi
ui(xi,xe−i)
subject to: xi ∈ x1i , . . . , x
Kii
(2.15)
Due to the finite number of strategies, the payoff matrix of the game can
be constructed, where each strategy combination is evaluated at the payoff
function of each player. Algorithms for solving Nash equilibria from its payoff
matrix are well known [11]. An alternative way to construct the payoff matrix
is to solve the inequality system proposed in (2.16) by repeating the inequality
42 2. Mathematical Framework for Bilevel Games
equation for every available strategy of each player.
ui(xei ,x
e−i) ≤ ui(x
kii ,x
e−i), ∀ki = 1, . . . , Ki, ∀i = 1, . . . , n (2.16)
The i-th payoff at the equilibrium (left hand side of (2.16)) must be less
than or equal to the i-th payoff for any other available strategy for the i-
th player, when the rest of the players have no incentives to change their
strategies, i.e., when they are at the equilibrium. The inequality system has∑ni=1Ki inequalities instead of the
∏ni=1Ki elements of the payoff matrix.
Example 2.4 Based on the previous two-player game from Example 2.1, now
the strategy space for player 1 and player 2 is discretized in 6 and 7 lev-
els respectively. Therefore, player 1 can choose amongst the strategies x1 =
x11, x
21, . . . , x
61 and player 2 can choose amongst the strategies x2 = x1
2, x22, . . . , x
72.
The finite NEP for the two-player game is defined in (2.17). Figure 2.7
shows the discrete strategy space.
(xe1, xe2) solves
u1(xe1, xe2) ≤ u1(x1
1, xe2)
. . .
u1(xe1, xe2) ≤ u1(x6
1, xe2)
u2(xe1, xe2) ≤ u2(xe1, x
12)
. . .
u2(xe1, xe2) ≤ u2(xe1, x
72)
xe1 ∈ x11, x
21, x
31, x
41, x
51, x
61
xe2 ∈ x12, x
22, x
32, x
42, x
52, x
62, x
72
(2.17)
Assume that the gradients of the payoff functions are the same as in Exam-
ple 2.3, ∇u1(x1, x2) for player 1 and ∇u2(x1, x2) for player 2, and defined only
for the discretized strategies based on the original continuous case. The finite
NEP solution is unique and located at (xe1, xe2) = (x6
1, x72). It is represented with
a red dot in Figure 2.7. In this case, the solution from the finite NEP remains
2.3. One-Level Games 43
x2
x1
x12
x72
x11 x6
1x51x4
1x31x2
1
x22
x32
x42
x52
x62
∇u1(x1, x2)∇u2(x1, x2)
NE
Figure 2.7: Discrete strategy set and solution for the finite NEP
the same as in the original continuous problem. But the NEP solution from
the discretized game may be different from the solution of the original game.
However, the discretized game could be tractable for solving global equilibria
whereas the original computational problem is not tractable, or the payoff
functions are non-convex. In general, games with non-convex payoff functions
do not find global solutions for the NEP.
The smoothness and convexity properties of the payoff function are not
necessary for finding a global solution of the proposed model (2.16), since the
inequality system checks that the equilibrium strategy is better than or equal
to other available strategies for all finite values of each player.
Converting the finite NEP into an inequality system increases the number
of equations but solves the problem of having non-convex and non-smooth
payoff functions in order to set a global NEP solution. Besides, the inequality
system can be added as a set of constraints of a more complex hierarchical
optimization problem, as will be seen in Chapter 4.
2.3.6 Finite Generalized Nash Equilibrium Problem with
Shared Constraints
The discretization approach proposed above has limitations for the GNEP
because the set of inequalities must be evaluated for all the finite strategies of
44 2. Mathematical Framework for Bilevel Games
each player when the other players are in the equilibrium. In other words, all
the finite strategies xi ∈ x1i , . . . , x
Kii must be feasible given a fixed decision
vector x−i in the equilibrium, which is more restrictive than the conventional
definition of the GNEP. The latter forces feasibility only at the equilibrium
solution, i.e., xei . Therefore, a discretization of the GNEP entails a reduction
of the original feasible region and the equilibria may be different.
In the next example we clarify this fact for a GNEP with shared constraints.
Example 2.5 Based on the previous two-player game (Example 2.4), we have
added a new shared constraint over the set of strategies of both players, x1 and
x2 (see Figure 2.8).
The payoff functions’ gradients are ∇u1(x1, x2) for player 1 and ∇u2(x1, x2)
for player 2, as in the previous examples. Then, the set of solutions for the
(continuous) GNEP with shared constraints is represented by the thick red line.
Note that there is an infinite number of GNE.
Now, we have discretized the problem with the same levels as in the previous
Example, i.e, player 1 can choose amongst the strategies x1 = x11, x
21, . . . , x
61
and player 2 can choose amongst the strategies x2 = x12, x
22, . . . , x
72. The
equivalent finite GNEP is the same as in the previous example (2.17). We
assume the payoff function is known at each discrete combination of strategies,
based on the payoff gradients from the continuous problem. Then, player
1 chooses the highest values for their own strategies, x1, while player 2 is
interested in choosing the highest values of their own strategies, x2.
Assume that if the equilibrium decision of player 1 is xe1 = x61, then,
player 2 must evaluate the payoff function, u2, at all their finite available
strategies with xe1 = x61. But, for the cases when variable x2 takes the values
x62, x
72, the problem becomes infeasible. Therefore, xe1 = x6
1 can not be solution
of the problem (2.17). Then, the solutions of the discretized GNEP with
shared constraints are searched for in a reduced feasible region represented in
Figure 2.8 in dark color. This reduced feasible region constitutes an equivalent
standard NEP feasible region, in which the decision of each player is not
constrained by the decisions of the other players.
The solution of the discretized GNEP is represented with a red dot and it
2.3. One-Level Games 45
x2
x1
x12
x72
x11 x6
1x51x4
1x31x2
1
x22
x32
x42
x52
x62
∇u1(x1, x2)∇u2(x1, x2)
Discretized GNE
Set of continuous GNE
Reduced feasible region
Figure 2.8: Discretized GNE with shared constraints
differs from the original continuous GNEP.
Discretized GNEPs have limitations using the proposed approach, as we
have illustrated in the previous example. But they can succeed with other
problems like finding global solutions for the non-linear and non-convex payoff
functions, or finding all pure Nash equilibria as will be described in the next
subsection.
2.3.7 Finding All Pure Nash Equilibria in a Finite NEP
The finite NEP problem (2.16) may have a single solution, a manifold of finite
solutions, or may have no solution. It is important to know all the solutions
because, a priori, all equilibria are possible, or some of them are more meaning-
ful. But most of the proposed solution techniques do not guarantee finding all
pure Nash equilibria. Some of the algorithms find a single equilibrium without
any meaningful criteria.
We propose an algorithm for finding all pure Nash equilibria of a finite
NEP in (2.16). Due to the fact that there is a finite number of strategies for
each player, we can create “holes” in the feasible region for each identified
Nash equilibrium. A hole is represented with a new constraint and added to
the inequality system (2.16), so that the identified Nash equilibrium cannot be
a solution of the new inequality system. In this way we can find all pure Nash
46 2. Mathematical Framework for Bilevel Games
equilibria.
In order to do this, after a solution (Nash equilibrium) for the NEP (2.16)
(characterized by x∗i for all i) is found, we impose a new constraint to avoid
that the optimal value of xei being close to the previously found solution (within
a distance of ε). We repeat this procedure with any new solution found. Thus,
given a solution vector x∗i (q) of the NEP, we include a set of new constraints
to generate holes in the space of solutions already found, as described in (2.18)
for each Nash equilibrium found (indexed by q):
√∑i
(xei − x∗i (q))> · (xei − x∗i (q)) ≥ ε, ∀q (2.18)
Equation (2.18) represents q hyperspheres with radius ε > 0 centered at
point x∗i (q). Thus, the distance between xei and x∗i (q) must be greater than
radius ε. The left hand side of the equation represents the Euclidean distance
between these two points.
We can convert the integer variables into a new binary (0/1) representation,
using binary expansion [115, 116], for example. The number of variables
increases, but this helps to linearize equation (2.18). Nevertheless, some works
[116] have provided theoretical and computational evidence demonstrating that
transforming integer problems with binary variables helps to solve the problem
more efficiently with specific algorithms.
Assume the vector of strategies, xi, is a vector of binary variables that
represents all available strategies for the i-th player. Then, equation (2.18)
can be rewritten as (2.19).
∑i
((xei )
2 + (x∗i (q))2 − 2xeix
∗i (q)
)≥ ε2, ∀q (2.19)
where the quadratic term can be converted into a linear term (2.20) by
taking into account the properties of the binary variables.
2.4. Bilevel Games 47
∑i
(xei + x∗i (q)− 2xeix∗i (q)) ≥ ε2, ∀q (2.20)
The ε2 value must be small enough so as not to lose solutions inside the
hypersphere hole, and the solution must not belong to the boundary of the
hypersphere hole. Since the variables belong to the 0-1 discrete space, the
limits of ε2 are 0 < ε2 < 1.
2.4 Bilevel Games
Bilevel games are hierarchical games where players make decisions in sequence.
The simplest bilevel game is the so-called Stackelberg game [69] or single-leader-
single-follower game, where a leader makes decisions prior to the follower ’s
decisions.
As a generalization of the two-player Stackelberg game, new bilevel games
have been proposed in game theory literature. In these generalizations, the
lower and/or upper level have more than a single player. Thus, the players
at the upper level (leaders) make decisions simultaneously competing between
them and prior to the decisions of the players at the lower level (followers).
After the leaders make their decisions, the followers make their decisions, also
competing among themselves. The decisions of the followers are made taking
into consideration the leaders’ and the other followers’ decisions. Since a
follower competes against other followers, the lower-level problem forms a Nash
game parameterized in terms of the leaders’ decisions. In a similar manner, in
the upper-level problem, the leaders make simultaneous decisions considering
the optimal response of the followers. The leaders compete against each other
in the upper-level problem in a Nash game.
In bilevel games, leaders and followers can be different players or the same
players at both levels, but making different decisions. In Chapter 3 we pose a
bilevel game where the leaders are different players from the follower, but in
Chapter 4 we propose a hierarchical game where some players are playing at
the upper and lower levels making different decisions.
48 2. Mathematical Framework for Bilevel Games
Depending on the number of players at the upper or lower levels, bilevel
games can be classified into four categories: single-leader-single-follower, single-
leader-multiple-follower, multiple-leader-single-follower and multiple-leader-multiple-
follower game.
In general, bilevel games can be solved as bilevel optimization problems.
A work related with bilevel optimization [66, 117] can be applied for solving
bilevel games. When there are multiple players at the lower-level problem, the
problem can be rewritten as a set of equilibrium constraints in the optimization
problem of the leader(s). In case of a single leader, the problem is stated as an
MPEC optimization problem [67, 81]. If, instead, there are several players at
the upper-level problem, it can be stated as an EPEC optimization problem
[6,34,35].
2.4.1 Single-Leader-Single-Follower Games
A single-leader-single-follower game is stated as a bilevel optimization problem
[66,117]. The leader’s problem is at the upper level, where the leader chooses a
decision vector, x, first. After the leader has made their decision, the follower
chooses their decision vector, y, solving the lower-level optimization problem
(see Figure 2.9).
Leader
x y
Follower
Figure 2.9: Single-leader-single-follower game
The follower’s optimization problem is parameterized in terms of the upper-
level decision, x. Formally, the follower selects a vector, y(x), in some closed
set, Y , where their objective function is minimized, f(x, y). The optimal set
of solutions of the lower-level problem is denoted by S(x). Then, a vector
y(x) belongs to the optimal set of solutions of the lower-level problem, i.e.,
2.4. Bilevel Games 49
y(x) ∈ S(x), if and only if:
y(x) solves
minimizey
f (x, y)
subject to: y ∈ Y (x)
(2.21)
On the other hand, the leader minimizes their objective function, F (x, y),
in some closed set X, taking into account the optimal response of the follower,
y(x) ∈ S(x). This is formally described as follows:
(xe, ye) solves
minimize
x,yF (x, y)
subject to: x ∈ Xy ∈ S(x)
(2.22)
In this dissertation we investigate the case where the lower-level and upper-
level constraint functions are represented by linear functions. Therefore, the
lower-level constraint set, Y , is defined as Y = y : h(x, y) = 0, g(x, y) ≤ 0,where h(x, y) and g(x, y) are linear. The upper-level constraints set, X, is
defined as X = x : H(x, y) = 0, G(x, y) ≤ 0, where H(x, y) and G(x, y) are
linear.
Here, we have used the superscript e to represent the optimal solution for
the whole problem (upper and lower level). Additionally, we have extended the
conventional definition of bilevel problems including the Lagrange multipliers
from the lower-level to the upper-level objective function and constraints. In
this sense, the Lagrange multipliers solution from the lower-level can affect the
decisions of the leader.
Then, the single-leader-single-follower optimal solution is obtained by solv-
ing the problem (2.23)–(2.24).
50 2. Mathematical Framework for Bilevel Games
(xe, ye, λe, µe) solves
minimizex,y,λ,µ
F (x, y, λ, µ)
subject to:
G(x, y, λ, µ) ≤ 0
H(x, y, λ, µ) = 0
(y, λ, µ) ∈ S(x)
(2.23)
where (y, λ, µ) ∈ S(x) if and only if:
(y, λ, µ) solves
minimize
y,λ,µf(x, y)
subject to: g(x, y) ≤ 0, µ
h(x, y) = 0, λ
(2.24)
2.4.2 Single-Leader-Multiple-Follower Games
A single-leader-multiple-follower game is a Stackelberg problem extension with
multiple followers, where the followers are competing among themselves. Fig-
ure 2.11 represents the structure of this game.
In this game a single leader makes their optimal decision, x, prior to the
decision of multiple followers, who are competing among themselves. Given
the optimal decision of the leader, x, each j-th follower makes their optimal
decision, yj, taking into account their competitors’ optimal decisions, y−j.
Leader
Follower 1 Follower m
y1
y1
x x
ym
ym
Figure 2.10: Single-leader-multiple-follower game
The single-leader-multiple-follower equilibrium solution is given by solving
the problem (2.25)–(2.26). The vector (xe,ye,λe,µe) represents the optimal
2.4. Bilevel Games 51
values of the decisions of the leader and the followers, as well as the Lagrange
multipliers of the lower-level problem.
The leader minimizes their objective function, F (·), which depends on the
leader’s decision, x, the optimal decisions of the followers, y, and the optimal
value of the Lagrange multipliers, λ and µ, from the lower-level problem. The
upper-level problem (2.25) is constrained by the functions G(·), H(·) and the
set of the optimal solutions of the followers, S(x), parameterized by the leader’s
decision, x, solving a set of m problems in the lower level (2.26).
(xe,ye,λe,µe) solves
minimizex,y,λ,µ
F (x, y, λ, µ)
subject to: G(x, y, λ, µ) ≤ 0
H(x, y, λ, µ) = 0
(y, λ, µ) ∈ S(x)
(2.25)
where (y, λ, µ) ∈ S(x) if and only if:
(yj, λj, µj) solves,
∀j = 1, . . . ,m
minimizeyj ,λj ,µj
fj(x, yj, y−j)
subject to: gj(x, yj, y−j) ≤ 0, µj
hj(x, yj, y−j) = 0, λj
(2.26)
The y-tuple is the Nash equilibrium of the followers for the leader’s deci-
sion, x. The variables λ and µ represent the Lagrange multipliers for the
equality and the inequality constraints of the followers, respectively. The
objective function, fj(·), and the constraints, gj(·) and hj(·), are defined as
linear functions for all the j-th followers’ problems. Because each j-th follower
problem is stated as an LP, global optimality can be guaranteed for each j-
th follower problem. But the simultaneous j-th followers’ problems may not
have a solution, may have only one solution, or may have multiple solutions.
The set of the solutions represented by S(x) is rewritten sometimes as an
equivalent system of constraints, e.g., KKT conditions added to the upper-
level problem (2.25). This system of constraints is the so-called equilibrium
constraints set. The single-leader-multiple-follower problem can be stated as
52 2. Mathematical Framework for Bilevel Games
an MPEC optimization problem [67,81].
2.4.3 Multiple-Leader-Single-Follower Games
A multiple-leader-single-follower game is a case when several players (leaders)
anticipate simultaneously the decisions of a single player (follower). Because
all the leaders make decisions at the same stage, the upper-level problem is
defined as a Nash equilibrium of the leaders. Figure 2.11 illustrates the game
structure. Multiple-leader-single-follower games are appropriate for represent-
ing liberalized markets, where participants have to interact with the market
submitting offers prior to the resolution of the market. Market participants
are at the upper level and market operation is at the lower level. The Lagrange
multipliers of the lower-level problem represent on many occasions the price of
the resource traded in the market.
Leader 1
Follower
y
Leader n
yx1
x1
xn
xn
Figure 2.11: Multiple-leader-single-follower game
The formulation of the multiple-leader-single-follower game is given by
(2.27)–(2.28). Solving (2.27)–(2.28) means solving a set of n bilevel problems,
one per leader. Because all leader’s problems depend on the competitors’
decisions, the set of the n problems is coupled and complicates the resolution
of this problem. EPEC techniques [6, 34, 35] can be applied to solve this kind
of problem. Note that even though the lower-level problem is common for all
leaders, the response in primal and dual variables could be different. We have
emphasized this in the notation using the superscript (i) for the lower-level
variables.
2.4. Bilevel Games 53
(xei , ye, λe, µe) solves,
∀i = 1, . . . , n
minimizexi,y(i),λ(i),µ(i)
Fi(xi,xe−i, y
(i), λ(i), µ(i))
subject to:
Gi(xi,xe−i, y
(i), λ(i), µ(i)) ≤ 0
Hi(xi,xe−i, y
(i), λ(i), µ(i)) = 0
(y(i), λ(i), µ(i)) ∈ S(xi,xe−i)
(2.27)
where (y(i), λ(i), µ(i)) ∈ S(xi,xe−i) if and only if:
(y(i), λ(i), µ(i)) solves
minimizey(i),λ(i),µ(i)
f(xi,xe−i, y
(i))
subject to: g(xi,xe−i, y
(i)) ≤ 0, µ(i)
h(xi,xe−i, y
(i)) = 0, λ(i)
(2.28)
The lower-level problem (2.28) is an optimization problem parameterized
by the upper-level decisions of each of the i-th leaders. Because the lower-level
problem is a linear optimization problem it can be reformulated either as a set
of KKT conditions or as a set composed of the primal and dual constraints and
strong duality theorem. This set of equivalent constraints could be different
for each i-th leader’s problem, and is added to each upper-level problem.
Some authors [36] have claimed that when the lower level represents the
market operation, the Lagrange multipliers should be the same for all leaders,
i.e., λ(i) = λ, ∀i and µ(i) = µ, ∀i. This is known as price consistency,
where there is no price discrimination for all the leaders. A price-consistent
formulation is more restrictive and it may not have a solution while the original
one has. However, a price-consistent formulation is easier to solve than a
general one (2.27)–(2.28) due to the reduction in the number of variables and
constraints.
2.4.4 Multiple-Leader-Multiple-Follower Games
A multiple-leader-multiple-follower game is the most general instance of a
bilevel game where several leaders competing among themselves have to make
54 2. Mathematical Framework for Bilevel Games
decisions in the first stage prior to the decisions of a set of followers competing
among themselves in the second stage (see Figure 2.12).
Leader 1 Leader n
x1
x1
Follower 1 Follower m
y1
y1
xn
xn
ym
ym
Figure 2.12: Multiple-leader-multiple-follower game
The multiple-leader-multiple-follower problem is given by a set of n coupled
MPEC problems, one for each leader, and given by (2.29)–(2.30). This problem
is stated as an EPEC [6,34,35].
(xei ,ye,λe,µe) solves,
∀i = 1, . . . , n
minimizexi,y(i),λ
(i),µ(i)
Fi(xi,xe−i, y
(i), λ(i), µ(i))
subject to: Gi(xi,xe−i, y
(i), λ(i), µ(i)) ≤ 0
Hi(xi,xe−i, y
(i), λ(i), µ(i)) = 0
(y(i), λ(i), µ(i)) ∈ S(xi,x
e−i)
(2.29)
where (y(i), λ(i), µ(i)) ∈ S(xi,x
e−i) if and only if:
(y(i)j , λ
(i)j , µ
(i)j ) solves,
∀j = 1, . . . ,m
minimizey(i)j ,λ
(i)j ,µ
(i)j
fj(xi,xe−i, y
(i)j , y
(i)−j)
subject to: gj(xi,xe−i, y
(i)j , y
(i)−j) ≤ 0, µ
(i)j
hj(xi,xe−i, y
(i)j , y
(i)−j) = 0, λ
(i)j
(2.30)
2.4.5 Stochastic Multiple-Leader-Multiple-Follower Games
The perfect information hypothesis has been assumed in the previous Nash
game definitions. This means that all players, leaders and followers, have
perfect information about their competitors’ payoff functions, available strate-
gies and constraints. Additionally, all the exogenous parameters have been
2.4. Bilevel Games 55
assumed deterministic, but some of them could be random, such as demand or
cost. In this section we introduce stochasticity to bilevel games. In particular,
we expand the general case, the multiple-leader-multiple-follower game to a
stochastic game.
We assume the stochastic bilevel game is played in two stages. At the
first stage the leaders make their decisions in a Nash equilibrium setting,
prior to the knowledge of any scenario realization and considering the best
response of the followers. After the leaders make their decisions, the scenario
realization of the random vector, ξ, is known at the second stage and the
followers make their decisions in a Nash equilibrium setting. Therefore, the
lower-level equilibrium is solved for any realization of the random process,
ξ. Then, the set of (equilibrium) solutions from the lower level are random
variables in terms of such a random process. If we define symbol ξ as a
random distribution to model uncertainty, the lower-level variables are now
(y(i)(xi,xe−i, ξ), λ
(i)(xi,x
e−i, ξ), µ
(i)(xi,xe−i, ξ)) ∈ S(xi,x
e−i, ξ).
The stochastic multiple-leader-multiple follower optimization problem is
given by (2.31)–(2.32)
(xei ,ye,λe,µe) solves,
∀i = 1, . . . , n
minimizexi,y(i),λ
(i),µ(i)
E [Fi(·)]
subject to: Gi(·) ≤ 0
Hi(·) = 0
(y(i), λ(i), µ(i)) ∈ S(xi,x
e−i, ξ)
(2.31)
where (y(i), λ(i), µ(i)) ∈ S(xi,x
e−i, ξ) are given for the random distribution,
ξ, if and only if:
(y(i), λ(i), µ(i)) solves,
∀j = 1, . . . ,m
minimizey(i)j ,λ
(i)j ,µ
(i)j
fj(·)
subject to: gj(·) ≤ 0, µ(i)j
hj(·) = 0, λ(i)j
(2.32)
and where the variables from the second stage are defined as:
56 2. Mathematical Framework for Bilevel Games
y(i) = y(i)(xi,xe−i, ξ)
λ(i)
= λ(i)
(xi,xe−i, ξ)
µ(i) = µ(i)(xi,xe−i, ξ)
(2.33)
and the payoff and constraints functions are defined as:
Fi(·) = Fi(xi,xe−i, y
(i), λ(i), µ(i), ξ)
Gi(·) = Gi(xi,xe−i, y
(i), λ(i), µ(i), ξ)
Hi(·) = Hi(xi,xe−i, y
(i), λ(i), µ(i), ξ)
fj(·) = fj(xi,xe−i, y
(i)j , y
(i)−j, ξ)
gj(·) = gj(xi,xe−i, y
(i)j , y
(i)−j, ξ)
hj(·) = hj(xi,xe−i, y
(i)j , y
(i)−j, ξ)
(2.34)
The upper-level constraints and payoff functions are defined in terms of
expectations with respect to the random variable, ξ. Other kinds of con-
straints can be used as risk measures, but this is outside the scope of this
dissertation. The lower-level constraints and payoff functions are defined for
any realization of the random variable, ξ. When a scenario-based approach
is applied, the random variable ξ is sampled in the scenarios indexed by ω
and the real distribution is substituted by the scenarios’ realizations, ξ(ω).
Then, an equivalent deterministic optimization problem is obtained replacing
the random variable ξ by the sampled one, ξ(ω).
2.4.6 Stochastic Multiple-Leader-Multiple-Follower Games
in Finite Strategies
In this dissertation we have considered games with finite strategies for the lead-
ers and continuous strategies for the followers, including uncertainty modeling.
This game is cast as a finite stochastic EPEC. In general, it is a hard-to-solve
non-convex optimization problem.
In this Section we present a special case of this setting where the leaders’
decisions do not depend on their competitors’ decisions at the upper level.
2.4. Bilevel Games 57
This stochastic multi-leader-multi-follower game setting can model hierarchical
relationships among participants.
The mathematical model is stated as a stochastic EPEC in finite strategies
and it is defined in (2.35)–(2.36).
(xei ,ye,λe,µe) solves,
∀i = 1, . . . , n
minimizexi,y(i),λ
(i),µ(i)
E[Fi
(xi,x
e−i, y
(i), λ(i), µ(i), ξ
)]subject to: Gi(xi) ≤ 0
Hi(xi) = 0
xi ∈ x1i , x
2i , . . . , x
Kii
(y(i), λ(i), µ(i)) ∈ S(xi,x
e−i, ξ)
(2.35)
where (y(i), λ(i), µ(i)) ∈ S(xi,x
e−i, ξ) for the random distribution ξ, if and
only if:
(y(i), λ(i), µ(i)) solves,
∀j = 1, . . . ,m
minimizey(i)j ,λ
(i)j ,µ
(i)j
fj(xi,xe−i, y
(i)j , y
(i)−j, ξ)
subject to: gj(xi,xe−i, y
(i)j , y
(i)−j, ξ) ≤ 0, µ
(i)j
hj(xi,xe−i, y
(i)j , y
(i)−j, ξ) = 0, λ
(i)j
y(i)j ∈ RKj
+ , µ(i)j ∈ Rs
+, λ(i)j ∈ Rp
(2.36)
Leader i can choose among a finite number of strategies, xi ∈ x1i , x
2i , . . . , x
Kii ,
which are constrained by Gi and Hi functions. Note that the decisions of each
leader do not depend on the decisions of the other leaders (Gi and Hi are
dependent only on xi). In general, this is true in many problems of power
systems, where participants act in their own interests. However, the payoff
function only is dependent on the competitors’ decisions and the lower-level
variables. Additionally, the optimal decisions at the upper level are made
before the realization of the random variable, i.e., the decisions do not depend
on each single realization of the random variables.
At the lower level (equation (2.36)), follower j makes their optimal decisions
given the leaders’ decisions, (xi,xe−i), the other followers’ optimal decisions,
y−j, with full knowledge of the realization of the random variable, ξ = ξ(ω).
58 2. Mathematical Framework for Bilevel Games
This problem can be solved applying the approach explained in Section
2.3.6. Additionally, the algorithm for finding all Nash equilibria could be
applied, as described in Section 2.3.7.
Then, the stochastic EPEC in finite strategies can be cast as a set of a
system of inequalities with equilibrium constraints (2.37).
(xei ,ye,λe,µe) solves, ∀ki = 1, . . . , Ki, ∀i = 1, . . . , n
E[Fi(xei ,x
e−i, y
e, λe, µe, ξ
)]≤ E
[Fi
(xkii ,x
e−i, y
(i,ki), λ(i,ki), µ(i,ki), ξ
)]subject to:
Gi(xei ) ≤ 0, Gi(x
kii ) ≤ 0,
Hi(xei ) = 0, Hi(x
kii ) = 0,
xei ∈ x1i , x
2i , . . . , x
Kii ,
(ye, λe, µe) ∈ S(xei ,x
e−i, ξ), (y(i,ki), λ
(i,ki), µ(i,ki)) ∈ S(xkii ,xe−i, ξ)
(2.37)
The previous formulation represents a system of inequalities with equilib-
rium constraints where the minimization operator for the upper-level problem
does not appear. This is an advantage with respect to conventional EPEC
settings. Now, instead of having a set of n coupled MPECs, one for each
leader, we have a system of n inequalities with equilibrium constraints.
On the left hand side (LHS) of the inequality, the objective function of
leader i is evaluated at the equilibrium, xei , and the constraints must hold for
each i-th leader equilibrium, i.e., Gi(xei ) and Hi(x
ei ). Additionally, the equi-
librium constraints are solved for the leaders’ equilibrium, i.e., (ye, λe, µe) ∈
S(xei ,xe−i, ξ). We have added superscript e to the lower-level variables to repre-
sent the parametrization of the lower-level equilibria in terms of the upper-level
equilibria. Then, ye is the vector of the followers’ equilibria decisions when all
leaders are in the equilibrium.
On the right hand side (RHS) of the inequality, the objective function of
leader i is evaluated for each xkii available strategy. Each finite strategy for
leader i is restricted by their own constraints, Gi(xkii ) and Hi(x
kii ), and the
equilibrium constraints. The equilibrium constraints are parameterized for
2.4. Bilevel Games 59
each xkii strategy of leader i when the competitors are fixed in the equilibrium,
xe−i. We have added superscript (i, ki) to the lower-level variables to represent
the parametrization of the lower-level equilibria in terms of the upper-level
decisions. Then, y(i,ki) is the decision vector of the followers in the equilibrium,
when the i-th leader chooses the ki-th strategy and their competitors are in
the equilibrium.
The major challenge of this formulation is to solve the equilibrium con-
straints, since they are optimization problems nested in an inequality system.
In Section 2.5 we deal with these constraints, transforming the optimization
problems into their first-order optimality conditions.
For this particular finite stochastic EPEC, there is no relationship with
the competitors’ decisions in the upper-level constraints set, the same as in
a standard NEP. But there is an extra constraint, the equilibrium constraint
coupling the leaders’ decisions. This motivates the next subsection.
2.4.7 Bilevel Games could be Special Cases of General-
ized Nash Equilibrium Problems
Bilevel games are special cases of GNEPs. In particular, multi-leader games are
GNEPs because there are constraints in each leaders’ problem that involve vari-
ables of the other leaders. These constraints could be upper-level constraints
or equilibrium constraints. Regarding the upper-level constraints, they are
easy to understand when the problem is generalized, because the constraints
for each leader depend explicitly of the competitors’ strategies. Regarding the
equilibrium constraints, it is not easy to understand their dependence on the
competitors’s decisions, because they are implicit.
This implicit dependence on the competitors’ decisions in the lower-level
problem is usual when a common resource is traded or shared in the lower-
level problem, e.g., energy demand. Leaders can submit their desires to obtain
this resource by choosing their strategies, which, at first, are not restricted.
However, the resource is distributed among leaders at the lower level, where
the desires of the leaders are linked. Their distribution represents an implicit
60 2. Mathematical Framework for Bilevel Games
coupling constraint among the leaders.
We illustrate this fact in the next example, where the interdependence
among the leaders’ decisions only occurs in the equilibrium constraints.
Example 2.6 Given a multiple-leader-single-follower game with two leaders
and one follower, leader 1 chooses amongst strategies x1 ∈ X1 ⊆ R, leader
2 chooses amongst the strategies x2 ∈ X2 ⊆ R and the follower chooses the
strategies among y ∈ Y ⊆ R. The leaders’ decisions are not dependent on
each other’s decisions.
Let the objective functions be F1(x1, x2, y) : X1×X2×Y 7→ R for leader 1,
F2(x1, x2, y) : X1×X2×Y 7→ R for leader 2, and f(x1, x2, y) : X1×X2×Y 7→ Rfor the follower.
The multiple-leader-single-follower game is composed of the optimization
problems of the two leaders:
(xe1, xe2, y
e) solves (2.38)–(2.39)
minimizex1,y(1)
F1(x1, xe2, y
(1))
subject to: x1 ∈ X1
y(1)solves
minimizey
f(x1, xe2, y)
subject to: y ∈ Y
(2.38)
minimizex2,y(2)
F2(xe1, x2, y(2))
subject to: x2 ∈ X2
y(2)solves
minimizey
f(xe1, x2, y)
subject to: y ∈ Y
(2.39)
The feasible region for leader 1 is defined as Ω1(x1, xe2, y) = (x1, y) : x1 ∈
X1, y ∈ S(x1, xe2), and for leader 2 is defined as Ω2(xe1, x2, y) = (x2, y) : x2 ∈
X2, y ∈ S(xe1, x2). Then, the multiple-leader-single-follower game is written
in a short form in equation (2.40).
2.4. Bilevel Games 61
(xe1, xe2, y
e) solves
minimizex1,y(1)
F1(x1, xe2, y
(1)), s.t. (x1, y(1)) ∈ Ω1(x1, x
e2, y
(1))
minimizex2,y(2)
F2(xe1, x2, y(2)), s.t. (x2, y
(2)) ∈ Ω2(xe1, x2, y(2))
(2.40)
Figure 2.13 illustrates the set of available strategies for the leaders and the
follower as well as the feasible regions for both optimization problems (2.40).
The optimal solution of the lower-level problem has been assumed to be unique.
Then, for any leaders’ decisions vector, (x1, x2), the optimal response of the
follower is unique. The set Ω(x1, x2, y) provides the feasible region for the
leaders and the follower for any vector (x1, x2). The set Ω1(x1, xe2, y) represents
the feasible region for leader 1 and the follower, assuming leader 2 is fixed at
the equilibrium.
x2
x1
y
X2
Y
xe2
X1
Ω1(x1, xe2, y)
Ω(x1, x2, y)
Figure 2.13: Strategies set for players x1, x2 and y
Although the leaders are not restricted to any strategy, e.g. x1 ∈ X1, the
lower-level problem restricts the strategies that the leaders can choose (dark
62 2. Mathematical Framework for Bilevel Games
area in the x1–x2 plane from Figure 2.13). In this particular case, for si-
multaneous values of x1 and x2 close to zero, the problem becomes infeasible.
Therefore, there is no solution for the EPEC. For example, this constraint
could represent a resource that should be supplied at a minimum level, such as
electricity demand.
2.4.8 Other Bilevel Games Compositions
The basic element for bilevel games consists of leaders making decisions prior to
the followers’ decisions, both competing among themselves. We have pointed
out that when several players are competing at the same level, they are doing
it in a non-cooperative Nash equilibrium setting. This holds true in many
real situations where imperfect competition arises. But different kinds of
competitive behaviors could be included at each level, as in the case of perfect
competition. When markets are not concentrated or regulators restrict the
players’ behaviors, perfect competition should be expected.
For example, the problem of generation expansion in power systems could
be interpreted in this way: first, the leaders (GENCOs) decide their optimal
generation expansions in a Nash setting anticipating the results in the spot
market. Then, the spot market clearing process takes place at the lower level
and the participants (GENCOs and ISO) act in a perfectly competitive way.
2.5 Solving Bilevel Games
Bilevel games are highly non-linear and non-convex, thus, the existence and
uniqueness of equilibrium points rarely happens. For example, in the simplest
case, the single-leader-single-follower game is modeled as a bilevel game, gen-
erally NP-hard, i.e., no numerical solution scheme exists to solve the problem
in polynomial time [66].
MPECs and MPCCs are non-convex and non-linear and NLP algorithms
fail to solve such problems because the constraints qualification, such as LICQ
and MFCQ, fail with complementary constraints. Hence the global optimal so-
lution is seldom obtained. New constraint qualification definitions are proposed
2.5. Solving Bilevel Games 63
to define new stationarity solutions (not necessarily global solutions) reached
solving MPECs or MPCC by conventional NLP algorithms. For example,
the W-, C-, B-, M- or S- constraint qualifications are used. Reference [86]
defines such constraints qualifications for solving MPCC. See the monograph
on MPECs [81] for further details.
EPECs are composed of a set of coupled MPECs and they inherit the
“bad” properties of MPECs. They are non-convex and non-linear and finding
a solution for this problem constitutes a challenge. Thus, a global solution
is seldom reached. Because the constraint qualifications do not hold for each
MPEC that composes the EPEC, the solutions obtained are usually stationary.
These stationary solutions may be Nash equilibria, local equilibria or saddle
points.
Finding algorithms to solve this problem constitutes an ongoing line of
research. Two algorithms have been suggested in the literature for solving
EPECs:
• Diagonalization approach, by solving the MPECs of each player sequen-
tially until convergence. This approach can be further classified into two
methods, Jacobi and Gauss-Seidel method. See [19,93].
• Simultaneous solution method, by writing the strong stationary necessary
conditions for all MPECs and solving all the constraints simultaneously.
The solution of this problem is known as strong stationary solution.
See [36,95].
Because of the lack of a global solution for these approaches, some hybrid
methods pretend to find the “best” solution between different sets of solutions
found when the problem is solved with different starting points.
We have overcome all of these difficulties by converting the stochastic EPEC
into an system of inequalities with equilibrium constraints, transforming the
stochastic EPEC into an MILP. Consequently, global optimality is guaranteed
for the equivalent MILP at the expense of the tractability of the problem.
To solve bilevel games, a one-level reformulation is often used. First, we
attempt to replace the lower-level problem with their equivalent first order
64 2. Mathematical Framework for Bilevel Games
optimality conditions. Because the lower-level has been assumed to be an
LP problem, the KKT conditions are sufficient optimality conditions. An
alternative way is to replace the lower-level problem by the set of primal
constraints, dual constraints and the strong duality theorem, which constitute
a set of first order optimal and sufficient conditions.
After that, the equivalent equilibrium conditions are added as constraints
to each leader’s optimization problem, becoming a set of one-level problems
(MPECs) stated as an EPEC. In the stochastic version, the lower-level prob-
lem is solved for all scenario realizations, ω. One set of equivalent optimal
conditions per scenario is added to each leader’s problem, becoming a set of
stochastic MPECs or a stochastic EPEC.
2.5.1 Manifolds of Lower-Level Solutions
In order to have a unique solution, strict convexity of the lower-level problem
for each decision of the leader, x, and each realization, ω, are required. How-
ever, when the lower-level is linear (convex and concave at the same time),
KKT conditions are applicable, but a non unique (globally) optimal solution
is reached for at least one value of x. This means that, for a given decision of
leader x, the optimal decision of the follower is a set of decisions y(x, ω) with
the same objective function value. Then, the follower is indifferent to any of
their own decisions. In other words, the first order optimality conditions from
the linear lower level are sufficient, but there could be multiple solutions.
A bilevel solution is called an optimistic solution if the leader takes an
optimistic attitude towards the outcome of the follower. On the contrary, a
solution is called a pessimistic solution if the leader takes a pessimistic attitude
towards the outcome of the follower. If the problem has multiple followers,
the solution of the lower level could have multiple equilibria (solutions), and
optimistic or pessimistic solutions could be assumed by all the leaders. In
general, most bilevel games are implicitly formulated as optimistic.
2.5. Solving Bilevel Games 65
2.5.2 First-Order Optimality Conditions for the Lower-
Level Problem: KKT Conditions
For the sake of simplicity, we have defined a deterministic linear lower-level
problem for a single follower (2.41)–(2.43). The KKT conditions are derived
from this problem. For the case of multiple followers, equivalent lower-level
conditions are formed by each individual follower’s KKT conditions.
The linear lower-level problem is defined as:
minimizey
c>x+ d(x)>y (2.41)
subject to: Ax+B(x)y ≤ b1, µ (2.42)
Cx+D(x)y = b2, λ (2.43)
where µ and λ are the Lagrange multipliers (dual variables) associated with
the inequality and equality constraints. Then, we define the Lagrange function
L(y, µ, λ), respectively.
L(y, µ, λ) =c>x+ d(x)>y
−µ>(Ax+B(x)y − b1)− λ>(Cx+D(x)y − b2) (2.44)
We have omitted the dependence of the leader’s decision, x, in the Lagrange
function and the variables because the decision is a known parameter for the
lower-level problem.
Then, the KKT conditions are given by:
∇yL(y, µ, λ) = d(x)−B(x)>µ−D(x)>λ = 0 (2.45)
∇µL(y, µ, λ) = Ax+B(x)y − b1 ≤ 0 (2.46)
∇λL(y, µ, λ) = Cx+D(x)y − b2 = 0 (2.47)
µ>(Ax+B(x)y − b1) = 0 (2.48)
µ ≥ 0, λ : free (2.49)
66 2. Mathematical Framework for Bilevel Games
2.5.3 First Order Optimality Conditions for the Lower-
Level Problem: Primal, Dual and Strong Duality
Theorem
Given the linear lower-level problem defined as in (2.41)–(2.43), we can recast
it as an equivalent problem (2.50)–(2.52).
c>x+ minimizey
d(x)>y (2.50)
subject to: B(x)y ≤ b1 − Ax, µ (2.51)
D(x)y = b2 − Cx, λ (2.52)
Then, the associated dual problem [118] is defined as in (2.53)–(2.55).
c>x+ maximizeµ,λ
µ>(b1 − Ax) + λ>(b2 − Cx) (2.53)
subject to: B(x)>µ+D(x)>λ = d(x) (2.54)
µ ≥ 0, λ : free (2.55)
And the strong duality theorem [118] is defined as (2.56).
d(x)>y = µ>(b1 − Ax) + λ>(b2 − Cx) (2.56)
The set of primal constraints (2.57)–(2.58), dual constraints (2.59)–(2.60),
and the strong duality theorem (2.61) are equivalent to the KKT conditions
and, therefore, they are sufficient conditions for optimality.
Ax+B(x)y ≤ b1 (2.57)
Cx+D(x)y = b2 (2.58)
B(x)>µ+D(x)>λ = d(x) (2.59)
µ ≥ 0, λ : free (2.60)
2.5. Solving Bilevel Games 67
d(x)>y = µ>(b1 − Ax) + λ>(b2 − Cx) (2.61)
Notice that equations (2.57), (2.58), (2.59) and (2.60) are equivalent to the
KKT conditions (2.46), (2.47), (2.45) and (2.49), respectively. Both sets of
conditions differ only in the complementary condition (2.48) that appears in
the KKT conditions instead of the strong duality theorem (2.61) which appears
in the latter formulation. Therefore, both equations, (2.48) and (2.61) should
be equivalent. We derive the strong duality theorem from the KKT conditions
to prove that both sets of optimality conditions are equivalent.
From equation (2.45), we have:
B(x)>µ = d(x)−D(x)>λ (2.62)
By expanding (2.48) and substituting (2.62), we get:
Using the binary expansion approach, (3.11)–(3.12), and the strong duality
theorem (3.13) in the objective function (3.1), the g-th GENCO problem yields
the following stochastic MPEC problem stated as an MILP.
SMPEC-MILP
maxE [Ug(·)] =∑ω∈Ω
ρ(ω)
∑t
∑i∈Ig ,b∈B
[λofferibt qibt(ω) + ∆λibt
KΛibt∑k=0
2kzkibt(ω) (3.27)
qofferibt
ξibt(ω) + ∆qibt
KQibt∑k=0
2kwkibt(ω)− cibqibt(ω)
](3.28)
s.t.∑b∈B
(qofferibt
+ ∆qibt
KQibt∑k=0
2kykibt
)≤ Qi, ∀i ∈ Ig,∀t (3.29)
Linearized set of lower-level constraints (3.18)–(3.26) (3.30)
The decision variables of the problem (3.28)–(3.30) are: the binary variables
xkibt,∀i ∈ Ig, k, b, t and ykibt,∀i ∈ Ig, k, b, t from the upper-level problem,
the free variable πt(ω),∀t, ω; the positive variables qibt(ω),∀i ∈ Ig, b, t, ω and
ξibt(ω),∀i ∈ Ig, b, t, ω from the lower-level problem. Variables zkibt(ω) and
wkibt(ω) result from the linearization of the bilinear term of the upper- and
lower-level variables. Only two of the decision variables of the SMPEC-MILP
model are strategic variables (xkibt, ykibt). Both variables come from the binary
expansion approach of (qofferibt , λofferibt ).
All variables are controlled by the leader. The leader’s target is to an-
3.2. Spot Market Strategic Bidding Equilibrium 81
ticipate the reaction of the other GENCOs (which have fixed bids). If the
competitors behave as rational agents, they should choose their optimal bids.
Consequently, they choose the strategies that are the best ones against all the
other ones of their competitors (also assumed fixed); this represents the set of
(pure) Nash equilibria. Thus, we use the SMPEC model within an equilibrium
setting where the competitor’s strategies are fixed to the equilibrium values.
3.2.3 Stochastic EPEC MILP Formulation
The vector of strategies available for the g-th GENCO is defined as sg =
(xkibt, ykibt),∀i ∈ Ig, k, b, t. The stochastic Nash equilibrium [113] is defined
from the set of inequalities (3.31), for any feasible strategy vector s = (s1, . . . , sg, . . . , sG) ∈S. The feasible region S is defined with the set of constraints of the SMPEC-
MILP problem.
E[Ug(se1, . . . , s
eg, . . . , s
eG, ωg
)]≥
maxsg
E[Ug(se1, . . . , s
eg−1, sg, s
eg+1 . . . , s
eG, ωg
)], ∀g ∈ G (3.31)
The resulting problem (3.31) is a non-linear and non-convex set of inequal-
ities that represents a stochastic EPEC problem. In this setting, all GENCOs
solve their SMPEC-MILP problems simultaneously, and the fixed strategies
offers in prices and quantities result from the solution of the SMPEC-MILP
problem of the other GENCOs.
For each GENCO, the strategy vector sg = (xkibt, ykibt) consists of a discrete
set of bids where Mg = card(Mg) =∏
i∈Ig(2KΛibt+1 · 2Kqibt+1) is the available
number of combinations of the set of discrete strategies. The utility function is
evaluated in the inequality system for each discrete strategy mg ∈Mg,∀g ∈ G.
See [25] for further details.
E[Ug(se1, . . . , s
eg, . . . , s
eG, ωg
)]≥
E[Ug(se1, . . . , s
eg−1, s
mgg , seg+1 . . . , s
eG, ωg
)], ∀mg ∈Mg,∀g ∈ G (3.32)
82 3. Strategic Bidding in Electricity Markets
The problem set in (3.32) can be solved by simple enumeration of the
strategies available to each GENCO. In this set, the left-hand side (LHS)
represents the equilibrium point and the right-hand side (RHS) each available
strategy of the GENCO. The number of inequalities is given by∑
g∈GMg,
which is better than solving the combinatorial game by creating∏
g∈GMg
combinations in the payoff matrix.
The LHS expected profit of the GENCOs can be transformed, as shown in
(3.33), where the expected value is given by the linear objective function of
the SMPEC-MILP in the equilibrium.
E[Ug(se1, . . . , s
eg, . . . , s
eG
)]=∑
ω∈Ω
ρ(ω)
∑t
∑i∈Ig ,b∈B
[λofferibt qeibt(ω) + ∆λibt
KΛibt∑k=0
2kzekibt(ω)
qofferibt
ξeibt(ω) + ∆qibt
KQibt∑k=0
2kwekibt(ω)− cibqeibt(ω)
], ∀g ∈ G (3.33)
where the feasibility region of the LHS is the set of constraints (3.34)–
(3.44). This constraint set is the same as the one of the SMPEC-MILP, but
for the Nash equilibrium in this case (superscript e).
∑b∈B
(qofferibt
+ ∆qibt
KQibt∑k=0
2kyekibt
)≤ Qi ∀i ∈ I,∀t (3.34)∑
i∈I,b∈B
qeibt(ω) = dt(ω) ∀t,∀ω (3.35)
qeibt(ω) ≤ qofferibt
+ ∆qibt
KQibt∑k=0
2kyekibt ∀i ∈ I, ∀b, ∀t, ∀ω (3.36)(λofferibt + ∆λibt
−ei and ηξei , respectively, and considering (4.52)–(4.53) for
all i ∈ N inv.
The RHS in (4.59) is the utility function of each GENCO given a particular
value of the strategic decision variable. That is, we consider that GENCO G
chooses strategy sG (which involves investing in generation capacity at node
i up to the capacity gsGi , with i ∈ N invG ,∀G ∈ G), the definition of the utility
function for GENCO G is given by (4.61) 1 .
1Note that, since gsGi is known, it is possible to directly replace its value in equations(4.33)–(4.45), (4.52)–(4.53) and (4.55)–(4.56), without having the non-linear term that
4.2. Transmission and Generation Expansion as a Three-Level Model 121
U sGG
(gsGi , ge−i : ∀i ∈ N inv
G ,∀ − i ∈ N inv−G)
=∑ω∈Ω
ρ(ω)
∑i∈N fix
G
g0i ξsGi (ω) +
∑i∈N inv
G
gsGi ξsGi (ω)
− ∑i∈N inv
G
Ki
(gsGi − g0
i
),
∀sG ∈ SG,∀G ∈ G (4.61)
subject to the corresponding constraints of level 3, which are: (4.33)–(4.45),
(4.52)–(4.53) and (4.55)–(4.56), but considering them ∀sG ∈ SG,∀G ∈ G,
From Table 4.9, we observe that there are three optimistic optimal solu-
tions: investing 0.4 MW in line 1, investing 0.4 MW in line 2, and investing
0.4 MW in line 4. These three solutions have identical total costs, which
is a consequence of the similarity in the production cost function of all the
generation units (which leads to the same LMPs in all the nodes).
All case studies have been solved using CPLEX 11 under GAMS [119]. We
have used a Dell PowerEdge R910 x64 computer with 4 processors at 1.87 GHz
and 32 GB of RAM. Table 4.10 shows the running times and computational
complexity required for solving the problems. The second to fifth columns
show the 3-node network CPU times and the computational complexity for
the cases shown in Table 4.6. The sixth column shows the CPU time and the
computational complexity to solve the 4-node network case.
4.7. Case Study: The Sistema Interconectado Central (SIC) in Chile 137
Table 4.10: CPU times and computational complexity of the 3- and 4-nodenetworks
3-node network4-node network
l1 l2 l3 l1, l2, & l3
CPU time 7.58 s 11.74 s 13.06 s 267.26 s 1 h 25 min# of binary variables 607 607 607 667 1316# of positive variables 670 670 670 672 1152# of free continuous variables 2103 2103 2103 28563 200260# of inequality constraints 14603 14603 14603 155727 1077204# of equality constraints 5834 5834 5834 85214 599954
4.7 Case Study: The Sistema Interconectado
Central (SIC) in Chile
We illustrate the proposed model using a stylized representation of the main
Chilean power network, i.e., the Sistema Interconectado Central or SIC, as
shown in Figure 4.9. The SIC is a system composed of both generation plants
and transmission lines that operates to meet most of the Chilean electricity
demand. The SIC extends over a distance of 1,740 km covering a territory of
326,412 sq. km, equivalent to 43% of the country, where 93% of the population
lives. At the end of 2010, the SIC had 12,147 MW of installed power capacity,
54.5% thermal and 44.1% hydroelectric, while the annual gross generation of
energy was around 41,062 GWh [120]. Although the decision framework spans
a lifetime of 25 years, we have considered a one-year horizon in our model,
with annualized investment costs. Nevertheless, our model can be run for
investment decisions in a year-by-year fashion. As shown in Figure 4.9, the
SIC has 34 buses and 38 transmission lines. Four existing lines are candidates
for capacity augmentation and 2 new lines are candidates for construction
(represented by the dashed lines in Figure 4.9). There are four generation
companies (corresponding to the three major generation firms in Chile and
the rest are grouped into a fourth firm), each owning generation capacity at
multiple locations. The electric characteristics (i.e., resistance, reactance, and
thermal capacity rating) of the transmission lines of the network are obtained
from [120].
138 4. Transmission and Generation Expansion
1 Diego de Almagro 2202 Carrera Pinto 220
3 Cardones 220
4 Maitencillo 220
5 Pan de Azucar 220
6 Los Vilos 220
7 Quillota 2208 Polpaico 500/220 12 Rapel 220
9 San Luis 22010 A. Santa 220
11 Cerro Navia 220
13 Chena 220
14 Alto Jahuel 500/220
34 Loaguirre 220
16 Colbún 220
20 Ancoa 500/220
15 A. Jahuel 15417 Paine154
18 Rencagua 154
19 San Fernando 154
22 Parral 154
21 Itahue 220/154
23 Chillán 154
25 Charrúa 154
24 Charrúa 500/22026 Concepción 220/154
27 San Vicente 154
28 Hualpén 220/154
29 Coronel 154
30 Temuco 220
31 Valdivia 220
32 Barrio Blanco 220
33 Puerto Montt 220
Figure 4.9: Stylized representation of the Chilean SIC network
Data
The main data are summarized in Tables 4.11, 4.12 and 4.13 2. Appendix B
contains fulls detail of the SIC. Table 4.11 provides data for the candidate lines
included in the expansion planning. The third column shows the annualized
2It is worth mentioning that Chile does not use LMPs, but a type of ’regulated LMPs’.The Chilean National Energy Commission estimates every six months the projected averageLMPs for the next 48 months, using a stochastic dual dynamic program, and fixes themuntil the next revision as ’regulated nodal prices’ [121].
4.7. Case Study: The Sistema Interconectado Central (SIC) in Chile 139
transmission investment cost per MW installed. We have not considered
economies of scale. The fourth and fifth columns show the current capacity
and the maximum line expansion, respectively.
Table 4.12 provides data for the node candidates to expand. The second
and third columns show the name of the owner company and the technology
installed in the corresponding node. The installed capacity is shown in the
fourth column and the maximum expansion in the fifth column. The gen-
eration expansion is discretized by binary expansion in four equally-spaced
levels between the initial capacity and the maximum expansion capacity. For
example, the available expansion values for node 3 are [0, 48.5, 96.9, 145.4]
MW. Note that non-dispatchable (wind) generation has a small marginal cost
and there is no possibility of decreasing it, i.e. bi = 0. On the other hand, for
dispatchable generation, we have selected the same slope in the marginal cost
function based on the historical data of the investments in the SIC [120]. The
second and third columns from Table 4.13 provide information about the cost
parameters and the fourth column shows the actual value of the annualized
investment cost per MW installed in each node.
To make the model more realistic, we have limited the production to 30%
of the installed capacity for wind farms and 75% for hydro plants.
Hourly demand forecasts are obtained from [120] for a one-year period
(2010) and simplified into four demand scenarios: summer-peak, summer-off-
peak, winter-peak and winter off-peak.
Table 4.11: Line expansion data
From node To node
Annualized Current Maximumtransmission thermal limit thermal limit
Line investment decisions are continuous variables in the model. However,
a practical expansion project would require discrete expansion levels. For
example, an increase of 21 MW in the line connecting nodes 9 and 10 could
be achieved by changing the material of the conductor. On the other hand, an
expansion of 423 MW in the line connecting nodes 11 and 34 is big enough to
consider the construction of a 500 MW line.
The model has been formulated in the General Algebraic Modeling System
(GAMS) [119] and solved with CPLEX 11 solver in a Dell PowerEdge R910x64
computer with 4 processors at 1.87 GHz and 32 GB of RAM. Table 4.18 shows
the CPU times and computational complexities required for each case study.
Table 4.18: CPU times and computational complexity
Case 1 Case 2 Case 3 Case 4
CPU time 0.56 sec 26 h 58 min 8 h 46 min 12 h 11 min# of binary variables 584 23956 51990 53414# of positive continuous variables 720 29620 64196 65622# of free continuous variables 276 11360 24656 24656# of inequality constraints 1616 83876 188564 192852# of equality constraints 416 16936 36760 36760
144 4. Transmission and Generation Expansion
4.8 Summary and Conclusions
We have proposed a three-level MILP where a transmission planner decides on
the first level upon the best line investments, given the optimistic pure Nash
equilibria in generation investment (second level), and the market clearing
(third level).
The transmission expansion model anticipates decisions on generation in-
vestment, where the equilibrium on generation investment is modeled using a
stochastic EPEC. We propose an approximation of the line impedance values
as a function of the installed transmission capacities. This assumption allows
us to incorporate the changes in operation due to the new topology resulting
from line investments within the transmission planning level.
In this sense, if a transmission planner suggests building some lines in an-
ticipation to generation capacity investments, then this can induce generation
companies to invest in a more socially-efficient manner.
We also apply a new methodology to extract all possible pure Nash equi-
libria on generation investment by creating holes in the equilibrium solution
space.
The model is applied to 3- and 4-node illustrative examples and to a realistic
case study based on the main Chilean power system (SIC).
Chapter 5
Summary, Conclusions,
Contributions and Future
Research
This chapter summarizes this dissertation and its main conclusions. Then, the
most relevant contributions of this work are stated. Finally, future research
directions are suggested.
5.1 Thesis Summary
We have presented a mathematical framework for solving stochastic finite
bilevel games in restructured power systems and found all pure Nash equilibria
for such problems. These games are appropriate for studying strategic behavior
in restructured power systems, since they can model hierarchical competition
among the participants. We have shown the applicability of our framework
in both operations and planning. Regarding the operational framework, we
have formulated, solved and illustrated the strategic bidding problem. Re-
garding the planning framework, we have formulated, solved and illustrated
the transmission and generation expansion problem with a realistic case study.
The principal features outlined in this thesis are described below:
• First, we have described the main features of restructured power systems
145
146 5. Summary, Conclusions, Contributions and Future Research
and electricity markets. Also, we have summarized current literature
related to this thesis, namely, the strategic bidding problem, the trans-
mission and generation expansion problem, and equilibrium modeling in
electricity markets.
• Second, we have introduced game theory definitions related with one-
and two-level games. We have also described a methodology for finding
all Nash equilibria in finite-strategy games. Then, we have presented
the conversion of an EPEC into an inequality system with equilibrium
constraints due to the linear properties of the lower-level problem and
the existence of finite strategies of the leaders. The suggested linear
reformulation approach has enabled us to find global solutions.
• Third, we have proposed a bilevel game in an operational framework to
solve the strategic bidding problem. In this problem, GENCOs submit
their offers in quantities and prices to the ISO. Demand uncertainty is
considered in the lower-level problem. Therefore, each GENCO faces an
MPEC and the joint solution of all MPECs is stated as an EPEC. The
EPEC is reformulated as an MILP suitable to be solved with commercial
solvers. We have proposed two models: one with network representation
and another one disregarding the network. We have illustrated our
formulation with several case studies.
• Fourth, we have posed a three-level model within a planning framework
for transmission and generation capacity expansion. In this sense, the
transmission planner optimizes the expansion of the transmission net-
work (in the upper level) in anticipation of generation capacity invest-
ments (in the mid level). In the mid level, GENCOs compete among
themselves producing several Nash equilibria. Transmission and gener-
ation expansions are optimized in anticipation of the lower level, the
results of the spot market. The mid- and lower-level problems are stated
as stochastic EPECs in finite strategies. Then, a linearization process
to convert the original problem into an MILP is described. Also, a
methodology for finding all pure Nash equilibria in the mid level is
5.2. Conclusions 147
presented. Finally, an illustrative example and a realistic case study
based on the main Chilean power system (SIC) shows the applicability
of the proposed model.
5.2 Conclusions
The previous summary leads to relevant conclusions that can be drawn from
the research presented in this thesis. The most relevant conclusions are enu-
merated below:
1. The conclusions pertaining to bilevel games and their resolution are:
(a) Current algorithms for solving bilevel games have several shortcom-
ings. These algorithms are related with bilevel, MPEC or EPEC
optimization problems, whose solution methods have deficiencies.
(b) EPEC optimization problems lack a generalized theory for solving
them because, in general, such problems are non-linear and non-
convex and do not hold any constraint qualification. Specific meth-
ods have been proposed for solving specific instances of EPECs.
(c) Global solutions are rarely reached in EPECs. Instead of a global
solution, stationary solutions for EPECs are obtained. Such station-
ary solutions may be global, local, or saddle points. We have solved
this problem convexifying the EPEC and transforming it into an
MILP. The global solutions of the linearized problem are tractable
in moderately-sized optimization problems.
(d) In general, uniqueness is not guaranteed for EPECs. In fact, a man-
ifold of equilibria is a feature of many EPECs, but most algorithms
for solving EPECs only provide a single solution. We have solved
this problem including a new linear constraint that represents a
“hole” in the feasible region around each known Nash equilibrium.
2. The conclusions pertaining to the strategic bidding problem are:
148 5. Summary, Conclusions, Contributions and Future Research
(a) The bidding problem faced by a GENCO in an electricity spot
market can be formulated as a stochastic EPEC, modeling the
strategic behavior in a competitive setting. The problem can be
recast as a MILP. In this context, the bidding strategies and the
spot market prices are obtained from the solution of the proposed
model.
(b) The strategic bidding problem has multiple equilibria which can be
classified either by the expected profits obtained by the GENCOs
or by the expected spot market prices.
(c) In general, a network-constrained (and congested) model has less
equilibria than a network-unconstrained model. This is because the
GENCOs have more difficulties for finding a position where they
do not want to change their strategies. This happens when they
cannot deliver energy to the network without congesting any lines.
(d) The network-constrained (and congested) model provides the high-
est spot prices due to line congestion that may not enable full
dispatch of the cheap generating units.
(e) The network-constrained model equilibria without congestion are
also the same as the network-unconstrained model equilibria.
(f) The network-constrained case studies show that a GENCO may
congest the network as a mechanism to exert market power.
3. The conclusions pertaining to the transmission and generation capacity
expansion problem are:
(a) An anticipative transmission plan is important because it may in-
duce more socially-efficient (and/or environmentally-convenient) gen-
eration capacity investments.
(b) The anticipative model may help to mitigate the market power
exercised by GENCOs through their generation capacity investment
decisions, i.e., the transmission planner can trigger the construction
of some lines in anticipation of generation capacity investments,
5.3. Contributions 149
inducing generation companies to invest in a more socially-efficient
manner.
(c) Multiple generation expansion equilibria are possible for a single
transmission expansion plan, and all of them are perfectly valid.
The transmission planner can be modeled as an optimistic agent
if it anticipates the best EPEC equilibria from the social welfare
viewpoint.
(d) Finally, although the proposed methodology has several advantages,
it is important to recall its limitations. First, the model used
is static. This fact does not represent the dynamic nature of in-
vestments, but this assumption is made due to tractability issues.
Secondly, we assume perfect competition and inelastic demand (in
order to deal with convex problems), but the reader should be aware
of the possibility that GENCOs may exercise market power. And
thirdly, our model considers that transmission capacity investments
are continuous variables, although they are lumpy due to economies
of scale. In this sense, the numerical results of our model should
be taken as approximations of the transmission capacities to be
added to the network in order to produce the desired response by
generation capacity investors.
5.3 Contributions
The main contributions of this work can be summarized as follows.
1. Regarding the one- and two-level games, we have:
(a) Characterized and ranked the one- and two-level games configura-
tions as well as the mathematical formulation of each one of them.
(b) Proposed an MILP reformulation of a finite stochastic EPEC in
order to obtain global solutions.
(c) Developed a methodology for finding all pure Nash equilibria in
finite-strategy games.
150 5. Summary, Conclusions, Contributions and Future Research
2. Regarding the strategic bidding problem, we have:
(a) Formulated a bilevel game focusing on the strategic price and quan-
tity bidding variables of the GENCOs in a multi-period and multi-
block (bid) setting. In addition, we have considered stochasticity
of the demand in several scenarios. The bilevel game is stated as a
stochastic EPEC with finite strategies.
(b) Transformed the non-linear and non-convex stochastic EPEC into
an MILP.
(c) Found all the pure Nash equilibria of the stochastic EPEC by adding
new successive linear constraints to the linearized problem.
(d) Formulated the stochastic EPEC problem for a network-constrained
system and transformed it into an MILP.
(e) Applied the proposed model to two illustrative case studies.
3. Regarding the transmission and generation capacity expansion problem,
we have:
(a) Formulated a three-level model that integrates transmission plan-
ning, generation investment, and market operation decisions.
(b) Transformed the three-level model into a one-level MILP, that can
be solved with commercial solvers.
(c) Proposed and applied a methodology to solve the optimal transmis-
sion expansion problem (anticipating both generation investment
and market clearing).
(d) Characterized the equilibria in generation investment made by de-
centralized GENCOs (which corresponds to the solution of an EPEC)
as a set of linear inequalities.
(e) Computed all possible pure Nash equilibria of the generation invest-
ment problem by creating holes in the equilibrium solution space.
(f) Developed a methodology to account for the variation of line impedances
and PTDFs as functions of the installed transmission capacities.
5.3. Contributions 151
(g) Illustrated the proposed model with two case studies: 3-node and
4-node systems.
(h) Analyzed a realistic case study based on the main Chilean power
system (SIC) to show the applicability of the model.
4. The publication of the following six papers related to this dissertation
in relevant SCI-indexed international journals. The second, fourth and
fifth papers are directly related with this dissertation and the other ones
are collateral works.
(a) D. Pozo, J. Contreras, A. Caballero and A. de Andres, “Long-
term Nash equilibria in electricity markets,” Electric Power Systems
Research, vol. 81, no. 2, pp. 329–339, 2011.
(b) D. Pozo, and J. Contreras, “Finding multiple Nash equilibria in
pool-based markets: A stochastic EPEC approach,” IEEE Trans-
actions on Power Systems, vol. 26, no. 3, pp. 1744–1752, 2011.
(c) F. J. Dıaz, J. Contreras, J. I. Munoz and D. Pozo, “Optimal schedul-
ing of a price-taker cascaded reservoir system in a pool-based elec-
tricity market,” IEEE Transactions on Power Systems, vol. 26, no.
2, pp. 604–615, 2011.
(d) D. Pozo, J. Contreras, and E. E. Sauma, “A three-level static MILP
model for generation and transmission expansion planning,” IEEE
Transactions on Power Systems, in press.
(e) D. Pozo, J. Contreras, and E. E. Sauma, “If you build it, he will
come: Anticipative power transmission planning,” Energy Economics,
in press.
(f) D. Pozo, and J. Contreras, “A chance-constrained unit commitment
with an n−K security criterion and significant wind generation,”
IEEE Transactions on Power Systems, accepted for publication.
152 5. Summary, Conclusions, Contributions and Future Research
5.4 Future Research Suggestions
Suggestions for future research resulting from the work reported in this disser-
tation are listed below. They are organized into three main groups. The first
one refers to possible advances in the strategic bidding problem, the second
one focuses on the transmission and generation capacity expansion problem,
and the third one refers to the improvement of the algorithmic solutions used
in this thesis and the economic significance of bilevel games extensions.
1. Regarding the strategic bidding problem:
(a) Bilateral or forward contract markets may be included in our mod-
els. The resulting equilibria may change, but the enhanced models
could be useful tools for GENCOs, regulators and market operators.
(b) Modeling risk-adverse GENCOs may be desirable, since finding
equilibria with risk-adverse participants is a current research chal-
lenge. This is related with the previous item (1a) and item (3c).
(c) Detailed modeling of non-dispatchable renewable energy at the lower-
level problem would be valuable.
(d) The model proposed is adequate for a power system where the units
are mostly thermal generators. Hydro generators require special
treatment for water usage and this has not been included in the
models. A future extension may include specific modeling of the
water opportunity cost and hydro-cascade resource equations.
(e) The pool-market model may be extended to short-term electricity
markets such as intra-day markets or balancing markets.
(f) Demand side bidding may be considered in the market clearing
process.
(g) We have assumed demand as the single source of uncertainty, but
production costs, unit failure rates or renewable energies could also
be modeled as uncertain.
2. Regarding the transmission and generation capacity expansion problem:
5.4. Future Research Suggestions 153
(a) Development of proactive models (anticipative) for transmission
and generation expansion to model the strategic behavior of trans-
mission companies profiting from transmission rights.
(b) Modeling the GENCOs’ strategic behavior in the third level (spot
market), where the GENCOs can exert market power submitting
strategic offers to the ISO.
(c) Accurate representation of non-dispatchable renewable energies at
the lower-level would be of interest.
(d) A planning horizon spanning from 20 to 30 years, where investments
could be done in any year.
(e) Consider an n−K security criterion in transmission and generation
expansion planning.
(f) A model for the strategic behavior of demand, allowing consumers
to adapt their consumption to the resulting market prices.
3. Regarding the mathematical tools used in this thesis and the economic
significance of the bilevel games extensions:
(a) It would be of interest to apply decomposition techniques, since
the division of the main problem into subproblems could make the
problem easier to solve and the CPU time could be lower.
(b) Developing specific solution techniques for EPECs as MILPs with
a finite number of strategies may be desirable.
(c) In this thesis, we have assumed that players have a risk-neutral
attitude. It would be interesting to generalize the Nash equilibrium
problem with risk-adverse players, where risk-adverse parameters
could be included.
Appendix A
Capacity Expansion
SEPEC-MILP Formulation
This appendix contains the transmission planning problem formulated as an
MILP, where the objective function of the transmission planner is subject to
the generation expansion equilibria and the spot market equilibrium.
Transmission planner objective function maximization