Optimal Switching Games for Emissions Tradinghelper.ipam.ucla.edu/publications/fin2010/fin2010_8953.pdf1 Cap-and-Trade: Producer Perspective 2 Switching Games 3 Correlated Equilibria

1 / 33

Emissions Trading Switching Games Numerics Conclusion

Optimal Switching Games for Emissions Trading

Mike LudkovskiDepartment of Statistics & Applied Probability

University of California Santa Barbara

IPAM, January 8, 2010

Ludkovski Switching Games

2 / 33


Outline

1 Cap-and-Trade: Producer Perspective2 Switching Games3 Correlated Equilibria in CO2 markets4 Numerical Illustrations5 Open Problems


3 / 33


Emissions Trading

Major new initiatives are underway to introduce CO2 cap-and-tradeschemes that will create new commodity markets.AB32 proposal in California; various federal proposals.The estimated size of the market is in the hundreds of billions or eventrillions of dollars.Key regulatory details are still unresolved and undergo active publicdebate.Crucial to understand the financial implications of these initiatives onenergy producers.


4 / 33


A New Commodity Market

Compared to existing markets, cap-and-trade is fundamentally different:A finite resource is initially allocated and subject to exhaustion.A well-defined horizon (e.g. 1 year) exists for each allocation.The permit prices converge to deterministic values as horizonapproaches.Price formation is driven by participant strategies: must be endogenousto any model.Game-theoretic aspects emerge in the emissions market.


4 / 33


A New Commodity Market

Compared to existing markets, cap-and-trade is fundamentally different:A finite resource is initially allocated and subject to exhaustion.A well-defined horizon (e.g. 1 year) exists for each allocation.The permit prices converge to deterministic values as horizonapproaches.Price formation is driven by participant strategies: must be endogenousto any model.Game-theoretic aspects emerge in the emissions market.


5 / 33


Effect on Producers

The foremost constituency affected by cap-and-trade would be energyproducers.The net profit of energy production would change from the spark-spreadto the clean spread.Commodity prices (input fuel, output fuel) are stochastic.Must take into account (dynamic) strategies of other participants.Feedback between production policies and carbon prices.


6 / 33


Related Literature

Real Options: Dixit and Pindyck (1994), Dockner et al. (2000).Analysis of Cap-and-Trade: Carmona et al. (2008,2009), Cetin andVerschuere (2008), ...Optimal Switching Problems (single-agent): Zervos (2003), Hamadèneand Jeanblanc (2005), M.L. and Carmona (2008, 2009), Hu and Tang(2008).Optimal Stopping Games: Ohtsubo (1987,1991), Shmaya and Solan(2004), Ferenstein (2005,2007), Ramsey and Szajowski (2008).Stochastic Differential Games: Bensoussan, Friedman, Hamadène,Lepeltier,...


7 / 33


Model Setup

We focus on the timing optionality within a real-options framework.Consider a duopoly – two producers (representing different sets of powerplants).Each one sells electricity into a stochastic market at price Pt .Need emission permits to produce. Must buy CO2 permits on the marketat price Xt .Take a reduced-form price-impact model for (Xt ) (do not explicitly modelthe remaining supply of permits).Simplify the strategy set: at each time epoch either produce, or stayoffline, ξi (t) ∈ {0,1}.Each producer’s policy influences changes in X ; ⇒ the schedulingdecisions of agents affect each other.Discrete-time model.


8 / 33


Basic Ingredients(Pt ) is exogenously given and is a discrete-time process with Gaussianincrements.At each instant t player i chooses emissions regime: ξi (t).

(Xt ) is another 1-dim. process, drift is controlled by ~ξ(t); Gaussianincrements correlated with those of (Pt ).Changes in ξi are expensive (fixed switching costs Ki,ξi (t−),ξi (t)) andinduce inertia and hysteresis.Net revenue is ψi (p, x , ~ξ) = (aiPt − biXt − ci )ξi (t).Fixed horizon T : expiration date of the permits.

Each admissible control pair ~ξ induces a probability law P~ξ of (Xt ) throughthe price impact mechanism. Work with the physical measures.Each producer optimizes

V i (0,p, x , ~ζ) = E~ξ[

T−1∑t=0

(ξi (t)(aiPt − biX

(ξ)t − ci )− Ki,ξi (t−),ξi (t)

)].


9 / 33


Dynamic Decision-Making

At each step, each producer decides whether to produce or not.The chosen action results in immediate date t-payoff, as well as differentcontinuation values on [t + 1,T ].Leads to a repeated 2× 2 stochastic game.Bellman’s Principle is replaced by a game Nash Equilibrium (NE).Pure Nash equilibria might not exist.Existence: Need mixed equilibria.Might also have multiple Nash equilibria.Uniqueness: equilibrium refinement.


9 / 33





9 / 33





10 / 33


Multiple Local Equilibria

Payoff functions: ψ1 = Pt − 2Xt − 10 (coal plant);ψ2 = 2Pt − Xt − 80 (clean gas plant).

The value functions are decreasing and convex in x around Xt = 10.Today Pt = 50,Xt = 15, both producers are offline but in-the-money.Tomorrow E[Pt+1] = 50.

Net profit is ∆Vi = E~ξ[Vi (Pt+1,Xt+1)]− Vi (p, x) + ψi (p, x).Expectations for tomorrow

Strategy ~ξ E~ξ[Xt+1] Net ∆~V(0,0) 15 (0,0)(0,1) 19 (−3,3)(1,0) 23 (2,−5)(1,1) 27 (−5,−6)


11 / 33


Classification of 2× 2 Games

Three equivalence classes:A single dominating pure equilibrium (unanimity).A competitive game (essentially zero-sum) which admits a unique mixedNash eqm.A (anti-) coordination game which admits two pure eqm’s, a mixed oneand a continuum of correlated eqm’s – “battle-of-the-sexes” as above.Profitable for each one to emit separately; not profitable to emit together.Which producer will yield??


12 / 33


Correlated Equilibria

Nash equilibrium: given the eqm strategy of the other player, maximizesyour expected payoff.Overall payoff distribution is a product measure on the payoff space.A correlated equilibrium (γ jk ) is a general probability distribution on thepayoff space. Known to all.Achieved by introducing a third (fictitious) agent, (regulator).The regulator sends a private signal µi (γ) ∈ {0,1} to player i .Given the signal (and implied strategy of the second player), no incentiveto deviate.


13 / 33


Formal 2× 2 Game

Payoffs H =

((α00, β00) (α01, β01)(α10, β10) (α11, β11)

).

A policy is (~π, ~ρ) whence πi (resp. ρj ) is the prob. that player 1 (player 2)chooses action i .

Value of a policy to players is Val(H;~π, ~ρ) :=

(∑i,j πiρjα

ij∑i,j πiρjβ

ij

).

γ = (γ ij ) is a CE if

{γ00α00 + γ01α01 ≥ γ00α10 + γ01α11, γ11α11 + γ10α10 ≥ γ11α01 + γ10α00

γ00β00 + γ10β10 ≥ γ00β01 + γ10β11, γ11β11 + γ01β01 ≥ γ11β10 + γ01β00.

Leads to game values Valγ(H) :=

(∑i,j γ

ijαij∑i,j γ

ijβ ij

).


14 / 33


More on Correlated Equilibria

The set of correlated equilibria includes the convex hull of all Nashequilibria.The correlation mechanism must be known in advance – in a multistagegame γ(t) can be Markovian in state variables.Common choices:

I A utilitarian mechanism, maximizing sum of game value for the agents.I An egalitarian mechanism, maximizing the minimum game value of the

agents;I A fixed preferential mechanism, maximizing the game value for player 1

(resp. player 2);

Given the signal, all actions are pure.


15 / 33


Stopping GamesA stopping game: each agent chooses a stopping time τi , i = 1,2.Stopping corresponds to action ’1’.Payoff structure (Z); agent i receives (τ ≡ τ1 ∧ τ2)

(τ−1∑t=0

Z 00i (t)

)+ Z 10

i (τ)1{τi<τj} + Z 01i (τ)1{τi>τj} + Z 11

i (τ)1{τi =τj}.

Starting with known values at T move back in time; each period yields a2-by-2 game with payoffs corresponding to conditional expectation ofnext-period value.Let Valγ(Zt ) be an equilibrium of a 2-by-2 one-period game with payoffs

Zt =

((Z̃1(t), Z̃2(t)) (Z 01

1 (t),Z 012 (t))

(Z 101 (t),Z 10

2 (t)) (Z 111 (t),Z 11

2 (t))

).

Stopping game values solve (V1(t),V2(t)) = Valγ(Zt ), with(Z̃1(t), Z̃2(t)) ≡ (E[V1(t + 1)|Ft ] + Z 00

1 (t),E[V2(t + 1)|Ft ] + Z 002 (t)).


15 / 33


Stopping GamesA stopping game: each agent chooses a stopping time τi , i = 1,2.Stopping corresponds to action ’1’.Payoff structure (Z); agent i receives (τ ≡ τ1 ∧ τ2)

(τ−1∑t=0

Z 00i (t)

)+ Z 10

i (τ)1{τi<τj} + Z 01i (τ)1{τi>τj} + Z 11

i (τ)1{τi =τj}.

Starting with known values at T move back in time; each period yields a2-by-2 game with payoffs corresponding to conditional expectation ofnext-period value.Let Valγ(Zt ) be an equilibrium of a 2-by-2 one-period game with payoffs

Zt =

((Z̃1(t), Z̃2(t)) (Z 01

1 (t),Z 012 (t))

(Z 101 (t),Z 10

2 (t)) (Z 111 (t),Z 11

2 (t))

).

Stopping game values solve (V1(t),V2(t)) = Valγ(Zt ), with(Z̃1(t), Z̃2(t)) ≡ (E[V1(t + 1)|Ft ] + Z 00

1 (t),E[V2(t + 1)|Ft ] + Z 002 (t)).


16 / 33


Nash Equilibria in Stopping Games

To show existence of a pure Nash equilibrium need restrictiveassumptions (e.g. Dynkin zero-sum games, non-zero-sum monotonegames where Z 01

i ≤ Z 11i ≤ Z 10

i ).In general, must allow randomized stopping times.This is an (Ft )-adapted stochastic process p = (pt ) with 0 ≤ pt ≤ 1 a.s.τ(p) , inf{t : ηt ≤ pt} where ηt ∼ Unif (0,1) i.i.d.. pt is the probability ofstopping at date t , conditional on not stopping so far.τ(p) is not (Ft )-adapted. Enlarge the filtration: τ(p) is a(Ft ∨ σ(ηt ))-stopping time.Shmaya & Solan (2004): any discrete-time stopping game admits amixed NE.Ferenstein (2005) gave a construction using backward recursion.Solution relies on recursive Nash equilibria and conditional expectations.


17 / 33


CE in Stopping Games

A correlation law (γ jk (t)) is a function of (t ,Pt ,Xt ) and fixed/known inadvance. Gives a CE for any payoff structure Zt .At each state t , agent i receives a private signal µi (t ; γ).Resulting randomized stopping time is τi (γ). τi , τj are dependent!At each stage γ jk (t) induces a CE – no incentive to deviate given µi (t ; γ).Admissible overall strategies are G i -stopping times τi , withG i

t = σ(Ps,Xs, µi (s),0 ≤ s ≤ t).Game is non-cooperative; no possibility of threats, etc. Even if deviatecontinue to receive future messages and no changes are made.


18 / 33


Switching Game I

We have a switching game. This is a sequential stopping game: canrepeatedly “stop” to alter production regimes in response to changingelectricity prices, permit prices or other agent’s actions.

Player i : value function Vi (t ,Pt ,Xt , ~ξt ).

(V1(t, ~ζ),V2(t, ~ζ)) =

Valγ

((E~ζ [V1(t + 1, ~ζ)|Ft ] + Z1(t),E~ζ [V2(t + 1, ~ζ)|Ft ] + Z2(t)) (V1(t, ζ1, ζ̄2),V2(t, ζ1, ζ̄2)− K2,ζ2 )

(V1(t, ζ̄1, ζ2)− K1,ζ1 ,V2(t, ζ̄1, ζ2)) (V1(t, ζ̄1, ζ̄2)− K1,ζ1 ,V2(t, ζ̄1, ζ̄2)− K2,ζ2 )

)

where Zi (t) , (aiPt − biXt − ci )ζi .Overall structure:

I Observe current state (Pt ,Xt , ~ξt−1);I Regulator carries out randomization;I Receive private signals µi(t ; γ);I Choose private actions;I Joint action ~ξt is revealed, update state variables for next period;


19 / 33


Switching Game II

Sketch of proof: Restrict strategy sets so that agents can only use up to(n,m) switches.

Translates into an iterative stopping game with payoffs corresponding to(n − 1,m), (n,m − 1) or (n − 1,m − 1) cases.

Fixing the strategy of one player; the other player solves a switchingproblem with respect to the enlarged filtration G i .

By definition of γ this gives a CE in the switching game.

Take n,m→∞ to obtain a coupled pair of value functions as above.


20 / 33


Digression: Single Player Case

Fix the strategy of one producer and consider the optimization of theother one.This becomes an optimal switching problem as studied in Carmona-M.L.(2008).The price impact leads to significant hysteresis effect.If the price impact is severe enough, will always stay offline (or at leastwith very high probability) – “blockading”.From player’s 1 perspective, the actions of player 2 are randomized:continuation values are unknown, optimal stopping in “randomenvironment”.Otherwise, standard optimal stopping problem in the enlarged filtration(G i

t ) that incorporates CE.


21 / 33


1 Emissions Trading

2 Switching Games

3 Numerics

4 Conclusion


22 / 33


Numerical Solution

To solve for the game values numerically need toI Be able to compute equilibria in 2× 2 games;I Compute conditional expectations.

Have explicit formulas for CE of 2× 2 games (answer depends on CEchoice).Need approximation; recall that (P,X ) have continuous space.

Need to work with four different prob. measures P~ζ due to the priceimpact.


23 / 33


Markov Chain Approximation

Method I: discretize the state space of (P,X ).Choose finite-state Markov (P̃, X̃ ) whose 1-step transitions are consistentwith those of (P,X ).Take a rectangular grid; allow (P̃, X̃ ) to have transitions only toneighboring grid points;Conditional expectations reduce to weighted sums;See the book by Kushner and Dupuis (2001).Generic proof of convergence of this approach for finite-actionnon-zero-sum stochastic game with 2 players was done in Kushner(2007).


24 / 33


Least Squares Monte Carlo

To compute the conditional expectations, another robust algorithm is touse Monte Carlo simulation.Simulate paths of (P,X ) for each of the four possible emission regimes ~ζ.Continuation values are approximated through a cross-sectionalregression.If the optimal decision is to switch to another regime, then use theapproximate continuation value; else recursively update the futurepath-value.Extends the Longstaff-Schwartz method for American option pricing (asingle optimal stopping problem).A single-agent switching problem was solved in Carmona-M.L. (2008).Straightforward extension to randomized stopping ... and to 2-playergame.


25 / 33


More on Simulation

Main challenge: to account for price-impact;

Values of Xt depend on chosen (in the past) ~ξ(t), but the algorithm isbackward in time;Solution: re-simulate forward Xt paths based on known forward gamevalues; update continuation values as realized payoffs in a simulatedsub-game on [t ,T ].

Everything must be done for each possible future regime ~ζ.


26 / 33


Example

Pt+1 = Pt · exp(2(50− log Pt ) + 0.4εPt );Xt+1 = Xt · exp(3(log(12 + 8ξ1(t) + 4ξ2(t)− log Xt ) + 0.25εXt ) withE[εPεX ] = 0.6;Revenues: Z1(t) = Pt − 2Xt − 10;

Z2(t) = 2Pt − Xt − 80;T = 1, 26 periods (∆t = 1/26); K ≡ 0.2.Using the simulation solver:

Correlation Law V1(0,P0,X0) V2(0,P0,X0)Utilitarian 5.30 4.14

Egalitarian 5.33 4.20Preferential 1 5.39 4.11Preferential 2 5.02 4.24


27 / 33


Local Equilibria

Optimal game strategy ξ∗ as a function of (Pt ,Xt ) for t = 0.25. Here ~ζ = (0, 0). The green region denotes theanti-coordination CE and the red region denotes the competitive mixed NE.


28 / 33


A Realized Equilibrium Path

Sample path of the controlled Xt , including the corresponding strategy ξ∗ ∈ {00, 01, 10, 11}. The top left panelshows the cumulative P&L of each player; the bottom left panel shows the raw P&L for each time period.

Finally, the right panel shows the evolution of the controlled Xt , as well as the implemented strategy (ξ1t , ξ

2t ).

Note as ξt increases, emissions rise and Xt tends to increase.


29 / 33


Conclusion

Stochastic games naturally occur in studying oligopolies.The emission market would be a new important class of such problems.Investigate the simplest possible scenario where the game is non-trivial:a new model of an optimal switching game.Already the problems of equilibrium-refinement and computationaltractability arise.


30 / 33


Open Problems

The permit price is endogenous to the duopoly problem – current priceshould be (a function of) conditional expectation of total future emissionof CO2.Are there any no-arbitrage restrictions in this market (depends on whatfinancial strategies participants may use)?Extend to a general equilibrium setup.Also, producers will be allowed to bank and trade their permits. How toincorporate initial permit allocations?


31 / 33


Continuous Time Formulation

Would like a continuous-time counterpart of the discrete-time model.There are no existing results on general sequential stochastic games.Here we have a natural structure – switching game as a sequence ofstopping games.Need to define/understand continuous-time CE?Randomized stopping times were considered in zero-sum context byTouzi and Vieille (2002).In our diffusion setup, should be related to solution of (obliquely reflected)high-dimensional BSDE.Such representations have been obtained in related stochastic differentialgames or zero-sum Dynkin games.Reflection rule is complicated – uses 8 different “continuation” processes!


32 / 33


References ICalifornia Air Resources BoardClimate Change Proposed Scoping Plan.http://www.arb.ca.gov, October 2008.

E. J. Dockner, S. Jorgensen, N.V. Long, and G. Sorger,Differential games in economics and management science.Cambridge University Press, Cambridge, 2000.

R. Carmona, M. Fehr, J. Hinz and A. Porchet.Market Design for Emission Trading Schemes,SIAM Review, forthcoming.

S. Hamadéne and J. Zhang.The continuous time nonzero-sum Dynkin game problem and application in game options, 2009.prepring, available at http://www-rcf.usc.edu/ jianfenz/Papers/HZ2.pdf.

H. J. Kushner.Numerical approximations for nonzero-sum stochastic differential games.SIAM J. Control Optim., 46(6):1942–1971, 2007.

Y. Ohtsubo.A nonzero-sum extension of Dynkin’s stopping problem.Math. Oper. Res., 12(2):277–296, 1987.

D. M. Ramsey and K. Szajowski.Selection of a correlated equilibrium in Markov stopping games.European J. Oper. Res., 184(1):185–206, 2008.


http://www.arb.ca.gov

33 / 33


References II

R. Carmona, and M. Ludkovski.Pricing asset scheduling flexibility using optimal switching.Appl. Math. Finance, 15(4):405–447, 2008.

M. LudkovskiStochastic Switching Games and Duopolistic Competition in Emissions Markets.In preparation, available on request.


Optimal Switching Games for Emissions Tradinghelper.ipam.ucla.edu/publications/fin2010/fin2010_8953.pdf1 Cap-and-Trade: Producer Perspective 2 Switching Games 3 Correlated Equilibria

Documents