Duality for linear programming. Illustration of the notion Consider an enterprise producing r items: f k = demand for the item k =1,…, r using s components:

Duality

for

linear programming

Illustration of the notion

• Consider an enterprise producing r items: fk = demand for the item k =1,…, r using s components: hl = availability of the component l = 1,…, s • The enterprise can use any of the n process (activities): xj = level for using the process j = 1,…, n cj = the unit cost for using the process j = 1,…, n

• The process j produces ekj units of the item k =1,…, r uses glj units of the component l = 1,…, s for each unit of its use


• Consider an enterprise producing r items: fk = demand for the item k =1,…, r using s components: hl = availability of the component l = 1,

…, s

• The enterprise can use any of the n process

(activities): xj = level for using the process j = 1,…, n cj = the unit cost for using the process j =

1,…, n • The process j produces ekj units of the item k =1,…,

r uses glj units of the component l = 1,

…, s each time it is used at level 1

• The enterprise problem: determine the level of each process for satisfying the without exceeding the availabilities in order to minimize the total production cost.

• Model

1

1

1

min

S. t. 1,2,..., (demands)

1,2,..., (availabilities)

0 1,2,...,

n

j jj

n

kj j kj

n

lj j lj

j

z c x

e x f k r

g x h l s

x j n


• A business man makes an offer to buy all the components and to sell the items required by the enterprise to satisfy the demands.

• He must state proper unit prices (to be determined) to make the offer interesting for the enterprise:

vk item unit price k = 1, 2, … , r

wl component unit price l = 1, 2, …, s.

vk

wl

1

1

1

min

S.t. 1,2,..., (demands)


0 1,2,...,

n

j jj

n

kj j kj

n

lj j lj

j

z c x

e x f k r

g x h l s

x j n


The business man must state proper unit prices (to be determined) to make the offer interesting for the enterprise

To complete its analysis, the enterprise must verify that for each process j, the cost of making business with him is smaller or equal than using the process j. But the cost of making business with him is equal to the difference between buyng the items required and selling the components unused in order to simulate using one unit of process j (cj ).

1 1

buying the selling the items components

r s

kj k lj l jk l

e v g w c

≤

1

1

1

min

S. t 1,2,..., (demands)


0 1,2,...,

n

j jj

n

kj j kj

n

lj j lj

j

z c x

e x f k r

g x h l s

x j n

vk

wl


• The business man problem is to maximize his profit while maintaining the prices competitive for the enterprise

1 1

buying the selling theitems components

r s

kj k lj l jk l

e v g w c

≤

1 1

1 1

max

S. t 1,2,...,

0 1,2,...,

0 1,2,...,

r s

k k l lk l

r s

kj k lj l jk l

k

l

p f v h w

e v g w c j n

v k r

w l s


• The enterprise problem: multiply the availability constraints by -1

1

1

1

min

S. t. 1,2,..., (demands)


0 1,2,...,

n

j jj

n

kj j kj

n

lj j lj

j

z c x

e x f k r

g x h l s

x j n

1

1

1

min

S. t. 1,2,..., (demands)


0 1,2,...,

n

j jj

n

kj j kj

n

lj j lj

j

z c x

e x f k r

g x h l s

x j n

1

1 1

1 1

max

S. t. 1,2,...,

0 1,2,...,

0 1,2,...,

r s

k k l lk l

r s

kj k lj l jk l

k

l

p f v h w

e v g w c j n

v k r

w l s

Enterprise problem

Business man problem

sj

j

rj

j

g

g

e

e

1

1

knkjkk eeee 21

1 2 lnl l ljg g g g

G

E

1

1

1

min

S.t. 1,2,..., (demands)


0 1,2,...,

n

j jj

n

kj j kj

n

lj j lj

j

z c x

e x f k r

g x h l s

x j n

T TE G

1 1j rj j sjkj lje e e g g g

r

s

n

n

r s

1

1

1

min

S. t. 1,2,..., (demands)


0 1,2,...,

n

j jj

n

kj j kj

n

lj j lj

j

z c x

e x f k r

g x h l s

x j n

1 1

1 1

max

S. t. 1,2,...,

0 1,2,...,

0 1,2,...,

r s

k k l lk l

r s

kj k lj l jk l

k

l

p f v h w

e v g w c j n

v k r

w l s

Primal

Dual

T T

T T

max

S. t.

, 0

vp f h

w

vE G c

wv w

x

Tmin S. t.

0

z c x

E fx

G hx

w

Tmin

S. t.0

c xAx bx

T

T

max S. t.

0

b yA y c

y

Primal dual problems

Linear programming problem specified with equalities

Linear programming in standard form

Tmin S. t.

0

c xAx b

x

Tmin S. t.

0

c xAx b

x

Primal problem Dual problem

Primal problem Dual problem

y x

y x

T

T

max S. t.

b yA y c

T

T

max S. t.

0

b yA y c

y

Tmin

S. t.

0

c x

Ax b

x

T Tmin 0

S. t.

0, 0

c x s

Ax Is b

x s

T

T

T

max

S. t. 0

b ycA y

I

T

Tmax

S. t. 0

b yA y c

Iy

T

Tmax

S. t. 0

b yA y c

y

Duality theorems

• It is easy to show that we can move from one pair of primal-dual problems to the other.

• It is also easy to show that the dual of the dual problem is the primal problem.

• Thus we are showing the duality theorems using the pair where the primal primal is in the standard form:

Tmin S. t.

0

c xAx b

x

primal Dual

T

T

max S. t.

b yA y c

Duality theorems

• Weak duality theorem

If (i.e., x is feasible for the primal problem) and

if (i.e., y is feasible for the dual problem), then

Proof Indeed,

0,: xbAxxx

T:y y A y c T Tb y c x

T T T T Tsince and 0.b y x A y x c A y c x

Duality theorems

• Corollary If and , and if

, then x* and y* are optimal solutions for the primal and dual problems, respectively..

Proof It follows from the weak duality theorem that for any feasible solution x of the primal problem

Consequently x* is an optimal solution of the primal problem.

We can show the optimality of y* for the dual problem using a similar

proof.

0,:* xbAxxx * T:y y A y c T * T *b y c x

T T * T *.c x b y c x

Duality theorems

• Strong duality theorem If one of the two primal or dual problem has a finite value optimal solution, then the other problem has the same property, and the optimal values of the two problems are equal. If one of the two problems is unbounded, then the feasible domain of the other problem is empty.

Proof The second part of the theorem follows directly from the weak duality theorem. Indeed, suppose that the primal problem is unbounded below, and thus cTx→ – ∞. For contradiction, suppose that the dual problem is feasible. Then there would exist a solution ,

and from the weak duality theoren, it would follow that ; i.e., bTy would be a lower bound for the value of the primal objective function cTx, a contradiction.

T:y y A y c T Tb y c x

Recall: The simplex multipliers

Denote the vector specified by

Then

or

where denotes the jth column of the contraint matrix A

mR

T T 1Bc B

T T Tc c A

Tj j jc c a

ja

T1 1 1, , , , , ,n n nc c c c a a

is the simplex multipliers vector

associated with the basis .B

The vector has one element associated

with each row (constraint) of the tableau.

T T T 1Bc c c B A

Duality theorems

To prove the first part of the theorem, suppose that x* is an optimal solution of the primal problem with a value of z*.

Let be the basic variables.

Let , and π be the simplex multipliers associated with the optimal basis. Recall that the relative costs of the variables are specified as follows

where denotes the jth column of the matrix A.

Suppose that the basic optimal solution has the following property

Consequently

mjjj xxx ,...,,21

T 1,2,...,j j jc c a j n

ja

T 0 1,2,...,j j jc c a j n

T 1,2,...,j ja c j n

T

1 2[ , ,..., ]B j j jm

c c c c

Duality theorems

Suppose that the basic optimal solution has the following property

Consequently

and the matrix format of these relations:

This implies that

i.e., π is a feasible solution of the dual problem.

T 0 1,2,...,j j jc c a j n

T 1,2,...,j ja c j n

TA c

T:y A y c

Tor 1,2,...,j ja c j n

Duality theorems

Determine the value of the dual objective function for the dual feasible

solution π. Recall that

It follows that

Consequently, it follows from the corollary of the weak duality theorem that π is an optimal solution of the dual problem, and that

T1 .BB c

T *.b z

T TT T 1 1 T * *( )B B B Bb b B c B b c x c z

Complementary slackness theory

• We now introduce new necessary and sufficient conditions for a pair of feasible solutions of the primal and of the dual to be optimal for each of these problems.

• Consider first the following pair of primal-dual problems.

TminS. t.

0

c xAx bx

primal Dual

T

T

maxS. t.

b yA y c x


• Complementary slackness theorem 1

Let x and y be feasible solution for the primal and the dual, respectively. Then x and y are optimal solutions for these problems if and only if for all

j = 1,2,…,n

Poof First we prove the sufficiency of the conditions. Assume that the conditions (i) et (ii) are satisfied for all j=1,2,…,n. Then

T

T

0

0j j j

j j j

i x a y c

ii a y c x

T[ ] 0 1,2,...,j j jx a y c j n

T

1

Hence 0n

j j j

j

x a y c

TminS. t.

0

c xAx bx

T

T

maxS. t.

b yA y c x


Consequently

and the corollary of the weak duality theorem implies that x et y are optimal solutions for the primal and the dual problems, respectively.

T T T T T T T

1 1 1

But 0n n n

j j j j j j j

j j j

x a y c x a y x c x A y c x b y c x

T[ ] 0 1,2,...,j j jx a y c j n

T

1

Hence 0n

j j j

j

x a y c

T Tb y c x

T T T T1 1 2 2

1

T1

T2

1 2

T

T T

, , ,

n

j j n nj

n

n

x a y x a x a x a y

a

ax x x y

a

x A y


Now we prove the necessity of the sonditions. Suppose that the solutions x et y are optimal solutions for the primal and the dual problems, respectively, and

Then referring to the first part of the theorem

T

T

Since 0 et 1,2,..., ,

it follows that 0 1,2,..., .j j j

j j j

x a y c j n

x a y c j n

T T T T T T T

1 1 1

0n n n

j j j j j j j

j j j

x a y c x a y x c x A y c x b y c x

T T .b y c x

The proof is completed.


• Now consider the other pair of primal-dual problems

• Complementary slackness theorem 2

Let x and y be feasible solution for the primal and dual problems, recpectively. Then x and y are opyimal solutions of these problems

if and only if

for all j = 1,2,…,n for all i=1,2,…,m

TminS. t.

0

c xAx bx

T

T

0

0j j j

j j j

i x a y c

ii a y c x

iii

iii

bxayiv

ybxaiii

0

0

TmaxS. t.

0

T

b yA y cy

y x


Proof This theorem is in fact a corollary of the complementary slackness theorem 1. Transform the primal problem into the standard form using the slack variables si , i=1,2,…,m:

The dual of the primal problem in standard form

TminSujet à

, 0

c xAx Is bx s

T T

T T

max maxS. t S. t.

0 0

b y b yA y c A y c

I y I y

TminS. t.

0

c xAx bx


Use the result in the preceding theorem to this pair of primal-dual problems

For j=1,2,…,n

and for i=1,2,…,m

TminS. t.

, 0

c xAx Is b

x s

T

T

0

0j j j

j j j

i x a y c

ii a y c x

00

00

ii

ii

syiv

ysiii

T

T

maxS. t.

0

b yA y c

I y

x

s

y


For j=1,2,…,n

and for i=1,2,…,m

and then the conditions become

T

T

0

0j j j

j j j

i x a y c

ii a y c x

00

00

ii

ii

syiv

ysiii

But i i is a x b

iii

iii

bxayiv

ybxaiii

0

0

TminS. t.

, 0

c xAx Is b

x s

Dual simplex algorithm

• The dual simplex method is an iterative procedure to solve a linear programming problem in standard form.

1 1 2 2

11 1 12 2 1 1

21 1 22 2 2 2

1 1 2 2

min

S. t.

...

...

. . . .

. . . .

...

0 1,2,...,

n n

n n

n n

m m mn n m

j

z c x c x c x

a x a x a x b

a x a x a x b

a x a x a x b

x j n


• At each iteration, a basic infeasible solution of problem is available, except at the last iteration, for which the relative costs of all variables are non negatives.

• Exemplemin 3/ 2 1/ 2 27

S. t. 1/ 4 1/ 4 6 / 4

1/ 4 3/ 4 15/ 2

1/12 5/12 13/ 2

, , , , 0

z u h

x u h

u p h

y u h

x y u p h

basic var . r.h.s.


Analyse one iteration of the dual simplex algorithm, and suppose that the current solution is as follows:

mic

njc

ij

j

,...,2,10

,...,2,10

basic var. r.h.s.

Leaving criterion

mic

njc

ij

j

,...,2,10

,...,2,10

basic var. r.h.s.

If 0 1,2, , , then the solution is feasible

and optimal. The algorithm stops.ib i m

Leaving criterion

mic

njc

ij

j

,...,2,10

,...,2,10

basic var. r.h.s.

1,2, ,

1

Otherwise, let min . If 0 1,2, , , then

the feasible domain is empty. Indeed, since

for all 0, 0 and

it is not possible to find 0 such that

r i rji m

n

rj j rj

rj

b b a j n

x a x b

x a

1

= . n

j rj

x b

Leaving criterion

mic

njc

ij

j

,...,2,10

,...,2,10

basic var. r.h.s.

1,2, ,

Otherwise, let let min 0. is the leaving variable.

The pivot will be completed with an element in the row.

rr i ji m

th

b b x

r

Leaving criterion

1 1s s

r rs s

m ms s

b a x

b a x

b a x

mic

njc

ij

j

,...,2,10

,...,2,10

basic var. r.h.s.

We select the entering variable in such a way that

1) the value of the leaving variable increases when

the value of increases

ii) the relative costs of all the variables remains non

nega

r

s

j

s

x

x

x

tive when the pivot on is completed to modify

the tableau. rsa

0rsa

Leaving criterion

mic

njc

ij

j

,...,2,10

,...,2,10

basic var. r.h.s.

0rsa





nega

r

s

j

s

x

x

x

tive when the pivot on is completed to modify

the tableau. rsa

1 2 10 1 0 0rnr r r

rs rs rs rs rs

aa a b

a a a a a

0,

1,2, ,

rjj s

rs

ac c

a

j n

Leaving criterion

0rsa

mic

njc

ij

j

,...,2,10

,...,2,10

basic var. r.h.s.





negat

s

r

s

x

x

x

ive when the pivot on is completed to modify

the tableau. rsa

1 2 10 1 0 0rnr r r

rs rs rs rs rs

aa a b

a a a a a

0,

1,2, ,

rjj s

rs

ac c

a

j n

If 0, then the value of can only increase

since 0 and 0.

rj j

s rs

a c

c a

Leaving criterion

0rsa

mic

njc

ij

j

,...,2,10

,...,2,10

basic var. r.h.s.





negat

s

r

s

x

x

x

ive when the pivot on is completed to modify

the tableau. rsa

1 2 10 1 0 0rnr r r

rs rs rs rs rs

aa a b

a a a a a

0,

1,2, ,

rjj s

rs

ac c

a

j n

For all such that 0, we have to inforce the non negativity

of the relative cost by selecting properly the pivot element .

rj

rs

j a

a

Entering criterion

For all such that 0, we have to inforce the non negativity

of the relative cost; i.e., rjj a

0, such that 0rjj s rj

rs

ac c j a

a

0, such that 0j srj

rj rs

c cj a

a a

1,2, ,1,2, ,

Then the index of the entering variable is such that

max : 0 or min : 0 j js srj rj

j nj nrs rj rs rj

s

c cc ca a

a a a a

, such that 0j srj

rj rs

c cj a

a a

Pivot

• To obtain the simplex tableau associated with the new basis where the entering variable xs remplaces the leaving variable xr we complete the pivot on the element 0.rsa

Exemple

• x is the leaving variable, and consequantly, the pivot is completed in the first row of the tableau

• h is the entering variable, and consequently, the pivot is completed on the element -1/4

• After pivoting, the tableau becomes

This feasible solution

is optimal

basic var.

basic var.

r.h.s.

r.h.s.

Convergence when the problem is non degenerate

• Non degeneracy assumption: the relative costs of the non basic variables are positive at each iteration

• Theorem: Consider a linear programming problem in standard form.

If the matrix A is of full rank, and if the non degeneracy assumption is

verified, then the dual simplex algorithm terminates in a finite number of iterations.

T minSubject to

0 , , matrix

n m

z c xAx b

xc x R b R

A m n

• Proof:

Since the rank of matrix A is equal to m, then each basic feasible solution includes m basic variables strictly positive (non degeneracy assumption).

But there is a finite number of ways to select columns among the columns of to specify an sub matrix of :

!

! ( )!

mn A m m A

nnm m n m

But the non feasible basis of are a subset of these. Then !

! ( )!

is an upper bound on the number of feasible basis of .

Ann

m m n mA

• The influence of pivoting on the objective function during an iteration of the simplex

→ scrs

r

a

b

Substact from

since 0, 0, and 0 under the non degeneracy ass.

rs

rs

r rs s

bz z z c z

a

b a c

z

basic var. r.h.s. Deviding row

by rs

r

a

Then and the value of the objective function increases stricly at each iteration.

Consequently, the same basic non feasible solution cannot repeat during the completion of the dual simplex algorithm.

Since the number of basic non feasible solution is bounded, it follows that the dual simplex algorithm must be completed in a finite number of iterations.

,z z

since 0, 0, and 0 under the non degeneracy ass.

rs

rs

r rs s

bz z z c z

a

b a c

Comparing(primal) simplexe alg. and dual simplexe alg.

Simplex alg.

Search in the feasible domain

Search for an entering variable to reduce the value of the objective function

Search for a leaving variable preservingthe feasibility of the new solution

Stop when an optimal solution is found or when the problem is not bounded below

Dual simplex alg.

Search out of the feasible domain

Search for a leaving variable to eliminate a negative basic variable

Search for an entering variable preservingthe non negativity of the relative costs

Stop when the solution becomes feasibleor when the problem is not feasible

Duality for linear programming. Illustration of the notion Consider an enterprise producing r items: f k = demand for the item k =1,…, r using s components:

Documents

primal dual slide

enterprise slide

use slide

unit of process j c

dual feasible solution

primaldual slide

model slide

pair of primal