A bit on the Linear Complementarity Problem and a bit about me (since this is EPPS) Yoni Nazarathy EPPS EURANDOM November 4, 2010 * Supported by NWO-VIDI Grant 639.072.072 of Erjen Lefeber
Jan 11, 2016
A bit on the Linear Complementarity Problemand a bit about me (since this is EPPS)
Yoni Nazarathy
EPPSEURANDOM
November 4, 2010
* Supported by NWO-VIDI Grant 639.072.072 of Erjen Lefeber
Overview• Yoni Nazarathy (EPPS #2):
– Brief past, brief look at future…
• The Linear Complementarity Problem (LCP)– Definition– Basic Properties– Linear and Quadratic Programming– Min-Linear Equations– My Application: Queueing Networks
Just to be clear: Almost nothing in this presentation (except for pictures of my kids), is original work, it is rather a “reading seminar”
Some Things From the Past
High School in USA
Primary School in Israel (Haifa)
Israeli Army
Israeli Army Reserve
Married
Divorced
Married AgainEmily Born
Kayley Born
Undergraduate Statistics/Economics
Masters in Applied Probability
Software Engineer in High-Tech
Industry
Ph.D with Gideon Weiss
Cycle Racing
Born 1974
Netherlands (Feb 2009 – Nov 2010)
Collaborations: Matthieu, Yoav, Erjen, Johan, Ivo, Gideon, Stijn, Dieter, Michel, Bert, Ahmad, Koos, Harm, Oded, Ward, Rob,
Gerard, Florin…
Yarden Born!!!
Nederlands: Ik dank dat het is heel gezelich om te pratten…
Raising young kids in Eindhoven:
HIGHLY RECOMMENEDED!!!
EURANDOM / Mechanical Engineering / CWI Amsterdam
Pedaling to see the Low Lands
``
Future in Oz…
Melbourne
Melbourne…
Maybe live here
Work here: Swinburne University
Also collaborate here: Melbourne University
Maybe also collaborate here: Monash University
Swinburne University of Technology
Looking for Ph.D Students…
What is driving my travels??Maybe fears of some things that can kill…
In the Middle East…
In the Netherlands
A slow death…
Australia must be a safe place….
Or is it?
In Summary…I hope to stay lucky, also in Oz…
Finally…The Linear Complementarity Problem
(LCP)
Definition,
( , ) :Find , such that,
,
0, 0,
' 0.
n n n
n
q M
LCP q M z w
w Mz q
w z
w z
The last (complemenatrity) condition reads:
0 0 and 0 0.i i i iw z z w
It’s all about Choosing a Subset…For {1,..., } denote by ( ) a matrix with
collumns taken from (identity matrix)
and collumns {1,..., } \ taken from .
n B
I
n M
is about finding and 0
such that
( )
In this case:
LCP x
B x q
0, .
0i
i ii
ix iw z
x ii
Illustration: n=2
1 0 11 20 1 2
1 12 11 20 22 2
011 11 2121 2
11 12 11 2
21 22 2
{1,2}:
{1}:
{2}:
:
qw w
q
m qw z
m q
m qz w
m q
m m qz z
m m q
1 11 12 1 1
2 21 22 2 2
1 0
0 1
w m m z q
w m m z q
{1,2}C
Complementary cones:
1
0
0
1
12
22
m
m
11
21
m
m
1
2
q
q
{1}C
{2}C
{ : ( ) , 0}C y y B u u
C
Immediate naïve algorithm with complexity 3 32 2n nn or n
Existence and UniquenessDefinition: A matrix, is a P-matrix if the
determinants of all (2 1) principal submatrices are positive.
n n
n
M
Theorem (1958): ( , ) has a unique solution
for all if and only if is a P-matrix.n
LCP q M
q M
11 22 11 22 12 21e.g.for 2 : 0, 0, 0n m m m m m m
P-matrix means that the complementary cones "parition" n
P-Matrixes
Symmetric Matrixes PD Matrixes
Relation of P-matrixes to positive definite (PD) matrixes:
Reminder(PD) :
' 0 0x Mx x
Reminder(PSD) :
' 0x Mx x
Computation (Algorithms)• Naive algorithm, runs on all subsets alpha• Generally, LCP is NP complete• Lemeke’s Algorithm, a bit like simplex• If M is PSD: polynomial time algs exists• PD LCP equivalent to QP• Special cases of M, linear number of iterations• For non-PD sub-class we (Stijn & Eren) have an
algorithm. Where does it fit in LCP theory?We still don’t know…
• Note: Checking for P-Matrix is NP complete, checking for PD is quick
2n
LCP References And Resources• Linear Complementarity, Linear and Nonlinear
Programming, Katta G. Murty, 1988. Internet edition.• The Linear Complementarity Problem, Second Edition, Richard
W. Cottle, Jong-Shi Pang, Richard E. Stone. 1991, 2009.• Richard W. Cottle, George B. Dantzig, Complementary Pivot
Theory of Mathematical Programming, Linear Algebra and its Applications 1, 103-125, 1968.
• Related (to queueing networks): Unpublished paper (~1989), Avi Mandelbaum, The Dynamic Complementarity Problem.
• Open problems in LCP…. I am now not an expert (but a user) .... So I don’t know…
• Gideon Weiss, working on relations to SCLP
Some Applications(and Sources) of LCP
Linear Programming (LP)
min '
. .
0
c x
s t Ax b
x
max '
. . '
0
b y
s t A y c
y
Primal-LP: Dual-LP:
Theorem: Complementary slackness conditions
min '
. .
, 0
c x
s t Ax b v
x v
max '
. . '
, 0
b y
s t u c A y
y u
Assume , , , are feasible for primaland dual:
0, 0 Theyareoptimalsolutionsi i i i
x v y u
x u y v
0 ',
0
c ALCP
b A
0 '
0
u A x c
v A y b
, , , 0u v x y
' 0u x ' 0v y
The LCP of LPFind:
Such that:
And (complementary slackness):
Lekker!
Quadratic Programming1
min ( ) ' '2
. .
0
Q x c x x D x
s t Ax b
x
Lemma: An optimizer, , of the QP also optimizesmin ( ) '
. .
0
c Dx x
s t Ax b
x
Proof:( )x x x x
( ) ( ) 0Q x Q x ( ' ) '( ) ( ) ' ( )
2c Dx x x x x D x x
x
QP-LP:
QP-LP gives a necessary condition for optimality of QP in terms of an checking optimality of an LP
QP:
0 1, Let be feasible.x
( ' ) '( ) 0c Dx x x
( ' ) ' ( ' ) 'c Dx x c Dx x
The Resulting LCP of QP
',
0
c D ALCP
b A
Allows to find “suspect” points that satisfy the necessary conditions: QP-LP
Theorem: Solutions of this LCP are KKT (Karush-Kuhn-Tucker) points for the QP
Corollary: If D is PSD then x solving the LCP optimizes QP.
Proof: Write down KKT conditions and check.
Note: When D is PSD then M is PSD. In this case it can be shown that the LCP is equivalent to a QP (solved in polynomial time). Similarly, every PSD LCP can be formulated as a PSD QP.
Our Application: Min-Linear Equations( )B
0
0
( ) '( ) 0
B
,w z ( ) ( )
0, 0
' 0
u I B v I B
z w
w z
( ( ) , )LCP I B I B
Find :
Open Jackson NetworksJackson 1957, Goodman & Massey 1984, Chen & Mandelbaum 1991
, ,M M M MP
1
'
( ') , ( ')
M
i i j j j ij
p
P
LCP I P I P
i
i
Traffic Equations:
i jp
1
M
1
1M
i jij
p p
Problem Data:
Assume: open, no “dead” nodes
Modification: Finite Buffers and Overflows Wolff, 1988, Chapter 7 & references there in & after
ii
Exact Traffic Equations for Stochastic System:
i jp
M
1
1M
i jij
p p
Problem Data:
, , , ,M M M M M M MP K Q
Explicit Stochastic Stationary Solutions:
Generally NoiK
MK1
1M
i jij
q q
i jq
11K
Generally No
Assume: open, no “dead” nodes, no “jam” (open overflows)
Traffic Equations for Fluid System
Yes
Traffic Equations
1 1
M M
i i j j ji j j jij j
p q
out rate
overflow rate ( ) ( )
1 1( ') ( ( ') ) , ( ') ( ')LCP I Q I P I Q I P
Wrapping Up
• LCP: Appears in several places (we didn’t show game-theory)
• Would like to fully understand the relation of our limiting traffic equations and LCP
• In progress paper with Stijn Fleuren and Erjen Lefeber, “Single Class Fluid Networks with Overflows” makes use of LCP theory (existence and uniqueness)
• I will miss EURANDOM and the Netherlands very much!• Visit me in Melbourne!!!
The End