Decomposition for optimzation.pdf

7/27/2019 Decomposition for optimzation.pdf

1/47

Introduction Reformulation Relaxation Decomposition based Methods Application Result

Application of Decomposition Based Optimization

Methods to MINLP problemsApplication to MINLP problems

Pratik Patil10302018

under guidance of

Ravindra Gudi&

Mani Bhushan

Pratik Patil

Decomposition Techniques for MINLP
http://find/http://goback/


2/47


Outline

1 Introduction

2 General Reformulation of MINLP

3 Relxation techniques for MINLP

4 Decomposition based Methods

5 Application

6 Result and Conclusions

Pratik Patil

http://find/


3/47


Mixed Integer Nonlinear Program (MINLP)

minx,y

f(x, y)

subject to g(x, y) 0;

h(

x, y) = 0;

x X, y Yinteger

Desired Properties:

f, g, h are smooth (convex) functionsX, Y are polyhedral sets, e.g. Y = {y [0, 1]p | Ay b}

y Y integer hard problem

f, c not convex Very hard problem

Pratik Patil

http://find/


4/47


Application of Decopmposition

Solving MINLP is hard due to combined integrality andnonlinearity

Consider integer variable y as complicating/interactingvariable and apply model coordination and linearization on it

(Model Coordination)This seperates MINLP into 2 subproblems:

NLP subproblem with y = yk for that iteration, commonlycalled primal problem.MILP subproblem which has its nonlinearities removed,

commonly called master problem.Various methods differ on formation of master problem.

Another type of decomposition is to separate sparse MINLPinto smaller subproblems. (Goal Coordination)

Pratik Patil



5/47


Primal Problem

Primal problem is similar for all decomposition based methodsIf Primal is infeasible for given yk, primal feasibility problem issolved

Primal P(yk)

minx

f(x, yk)

s.t. h(x, yk) = 0

g(x, yk)

0

x X Rn

Primal Feasibility-1

minxX

pi=1

i

s.t. h(x, yk) = 0

g(x, yk) i,

i 0

i = 1, 2, . . . , p

Primal Feasibility-2

minx

iI

wig+i (x, y

k)

s.t. gi(x, yk) 0

h(x, yk

) = 0x X i I

Primal povides UBD and cut information

Feasibility problem is used to exclude feasible point throughPratik PatilDecomposition Techniques for MINLP


6/47


Block-Separable Reformulation

The block-structure hi(x) =pk=1 h

ki (xJk) influences the

quality and computation of a relaxation

small blocks: fast computation of underestimators and cutslarge blocks: better relaxation (smaller duality gaps)

Many MINLP problems have a natural sparse structure whichcan be reformulated to block separable.

Pratik Patil


I d i R f l i R l i D i i b d M h d A li i R l
http://find/


7/47


Splitting-schemes

Sparse MINLP can be reformulated to block-separable with almost

arbitrary block-sizes by1 Partition the sparsity structure:

Esparse = (i,j) V2 |

2hl(x)

xixj= 0 for some l {0, . . . , m} and

and V = 1, . . . , n (Vertex set)

into blocks J1, . . . , Jp.

2 For each adjacent node set

Rk = {i

pl=k+1

Jl | (i,j) Esparse,j Jk},

add new variables yk R|Rk| and copy-constraints xRk = y

k

where k = 1, . . . , p.

Pratik Patil


I t d ti R f l ti R l ti D iti b d M th d A li ti R lt
http://find/


8/47


Extended block-separable reformulation

By replacing block-separable constraintspk=1

hi,k(xIk) 0

bypk=1

ti,k 0, gi,k(xIk, ti,k) := hi,k(xIk) ti,k 0, k = 1, . . . , p.

we obtain a problem with linear coupling constraints

min cTx + c0

s.t. Ax + b 0

gi,k(xJk) 0, i Mk, k = 1, . . . , p (Pext)

x [xIk, xIk]

(easier for generation and use of cuts)Pratik PatilDecomposition Techniques for MINLP

http://find/


9/47


Convex relaxation and Lagrangian relaxation

Convex relaxation of (Pext):

(C) min{cTx + c0 | x conv(G X) H}

Lagrangian relaxation of (Pext):

(D) max

minx{cTx + c0 +

T(Ax + b) | x conv(G X)}

Duality gap val(Pext) - val(D) smaller if blocks are larger

Pratik Patil


http://find/


10/47


1 Dual methods: Lagrangian DecompositionSolve (D) approximately by a subgradient method

2 Cutting plane methods:Solve (C) approximately by generating supporting hyperplanes

3 Column generation methods: Simplicial DecompositionSolve (C) approximately by generating extreme points andextreme rays.

Role of block-separation/Decomposition:Subgradients, supporting hyperplanes and extreme points arecomputed by solving small NLPs.

Pratik Patil


http://find/


11/47


Convex underestimator

Replacement of nonlinear functions hi by a convex underestimator

hi in (P) yields a nonlinear convex relaxation:

min h0(x)

s.t. hi(x) 0, i = 1, . . . , m

x [

x,

x]

-underestimators

f(x) = f(x) + , Diag(x x)(x x)

and 0, x [x, x]Polynomial underestimators

1 Generate underestimator, q(x) f(x), x [x, x] bysampling

2 Set q(x) = q(x) + , Diag(x x)(x x)

Pratik Patil


http://find/


12/47


Valid Cut addition

solve separation problem (small NLP):

k = min fk(xJk;)

s.t. gki (xJk) 0,

xJk [x, x],

and optimal variable value is xJkThe valid cuts added then are

Linearization cuts: gi,k(xJk)

T

(xJk xJk) 0, i AkLagrange cuts: LK(xJk; ) Dk() where

Dk() = minxGk LK(x; )

Level cuts: f(x) f(x) where x = pk=1xJk

Pratik Patil


http://find/


13/47

p pp

Methods employedMINLP problem

Generalized Benders decomposition

Outer Approximation

Extended cutting plane methodGeneralized cross decomposition

LP-NLP based Branch & Bound

Simplicial Decomposition

Lagrangian Decomposition

Pratik Patil


http://goforward/http://find/http://goback/


14/47

p pp

Generalized Benders Decomposition

Classical BD was for MILP, adapted to MINLP by Geoffrion

Has only one continuous variable, others are projected out

Applied to following class MINLP problems:

minx,y

f(x, y)

h(x, y) = 0

g(x, y) 0

x X Rn

y Y {0, 1}q

Pratik Patil


http://find/


15/47

Under following conditions:C1 : X is nonempty, convex and the functions, f(x, y), g(x, y)

are convex for each y {0, 1}q

and the functions h(x, y) arelinear for each y {0, 1}q.C2 : The set

Z = {z Rp | h(x, y) = 0, g(x, y) 0 for some x X}

is closed for each y Y.C3 : For each fixed y Y V where

V = { y | h(x, y) = 0 & g(x, y) 0 for some x X},

one of the following condition holds:1 the resulting problem has finite solution and has an optimal

multiplier vector for equalities and inequalities.2 the resulting problem is unbounded, that is its objective

function goes to

Pratik Patil


http://find/


16/47

Generalized Benders Decomposition

Primal Problem forkth iteration:

minx

f(x, yk)

s.t. h(x, yk) = 0

g(x, yk) 0

x X Rn

Master Problem for kth iteration:

minyY,B

B

s.t. B (y; k

, k

), k = 1, 2, . . . , K0 (y; l, l), l = 1, 2, . . . ,

where, (y; k, k) = minxX

L(x, y, , )

(y;

l

,

l

) = minxXL(x, y,

, )

Here L(x, y, k, k) = f(xk, yk) + kT

h(x, yk) + kT

g(x, yk),is Lagrangian of primal problem.

Pratik Patil


http://goforward/http://find/http://goback/


17/47

Generalized Benders Decomposition - Algorithm

General Procedure

Primal problem providesupper bound and Masterproblem provides lower

boundPrimal and master aresolved iteratively

Optimal solution involves

UBD LBD ,

0is small constant.

Pratik Patil


http://find/


18/47

Outer Approximation

Uses first order linearization instead of dual projection

Applied to following types of MINLP problem

Simple OA

minx,y cTy + f(x)

s.t g(x) + B(y) 0

x X, y Y

OA + EqualityRelaxation

minx,y

cTy + f(x)

s.t h(x) = 0

g(x) + B(y) 0

x X

y Y

Generalized OA

minx,y f(x, y)

s.t. g(x, y) 0

x X Rn

y Y = {0, 1}q

where

x X = {x | x Rn, A1

x a1} Rn

Y = x x 0 1 q A2 a2Pratik Patil


http://find/


19/47

Outer Approximation- General Procedure I

General Procedure of OA:let (xj, yj)solveNLP(yj)

linearize nonlinear f, g, habout (xj, yj)

new objective variable f(x.y)

MINLP(P) MILP(M)

(M)

minx,y cTy + OA

s.t. f(xk) + f(xk)(x xk)

0 g(xk) + g(xk)(x xk)

0 Tk[h(xk) + h(xk)T(x xk)]

x X, y YPratik PatilDecomposition Techniques for MINLP

http://find/


20/47

Outer Approximation - Master Problem

For non-convex f, g, h augmented penalty variant of OA is

used :

minx,y

cTy + OA +k

woksok +

i,k

wpi,kpi,k +

i,k

wqi,kqi,k

s.t. + sok f(xk) + f(xk)(x xk)

pk g(xk) + g(xk)(x xk)

qk Tk[h(xk) + h(xk)T(x xk)]

x X = {x | x Rn, A1x a1} Rn

y

Y= {

x|

x {0

,1}

q, A1

x

a2}

sok, pi,k, qi,k 0, k = 1, 2, ..., K

Master problem: lower bound (underestimate of f,g), primalproblem: upper boundStop if LBD UBD

Pratik PatilDecomposition Techniques for MINLP

http://find/


21/47

Generalized Cross Decomposition

Consists of two phases:

Primal and Dual subproblem phaseMaster problem phase

Condition on MINLP are same as GBD with extra conditionthat

minx

L(x, y, )

can be performed independently


http://find/


22/47

Generalized Cross DecompositionPhase-1

Phase-1 contains iterative solution of Primal and Dualproblems

Primal problem is equivalent to OA/GBD

Dual problem is given by:

minx,y

a1.f( x, y) + kT

1 g1( x, y) + a2.

s.t. g2( x, y) a2.

x X Rn

y Y = { 0, 1 }q

where a1 = 1; a2 = 0 for feasible primal

a1 = 0, a2 = 1 for infeasible primal


http://find/


23/47

Generalized Cross Decomposition IPhase-2

Two master problems

Primal Master Problem

miny,c

c

s.t. c q1

minxX

(q3( x, 1, 2)), y, 1, 2

;

0 q2minxX (q4(x, 1, 2)), y, 1, 2 ;y Y


http://find/


24/47

Generalized Cross Decomposition IIPhase-2

Relaxed Lagrange Master Problem

max1,c

c

s.t. c hk(1), k = 1, 2, . . . , K

0 hl(1), l = 1, . . . , L

1 0


Generalized Cross Decomposition - flowchart
http://find/


25/47

p

http://find/


26/47


Avoids re-solving MILP master problems

Consider MILP branch and bound

Nonlinear function linearized andcontinuous relaxation applied

Initial continuous solution branchesto start BB tree


http://find/


27/47


Interrupt MILP, when yk

ZI

isfound

Solve Primal NLP (yk) get xk


http://find/


28/47


Linearize f, c about (xk, yk)add linearization to the tree


http://find/


29/47


Continue MILP tree-search untilLBD UBD




30/47

Extended Cutting Plane Method (ECP)

Solves only MILP master problem in OA by linearization (cutting

plane) around feasible points

minZ

cTZ

S.t. l(k)j 0, j = 1, . . . , Lk,

AZ a, BZ = b,Z X Y, 1

where l (Z) = gi(zk) + .gi(z

k)T(z zk)

No primal (NLP) subproblem, instead Kelley cutting planemethod used

Slow nonlinear convergence

Each yk needs is checked against 3 conditions for optimality


http://find/


31/47

Extended Cutting Plane Method (ECP) I

1 C1: If lkJ(Zj)(atZ=Z

k

)

gj(Zk) Local Underestimator,

j = 1, . . . , Lk. If not then kj is updated as: ( > 1)

=

.kj , l

kj (z

k) > gj(zk),

kj , otherwise.

2 Case 2: If Zk is feasible then if,

kj g(Zj)

h Feasible Underestimators j = 1, . . . , Lk

is tolerance, otherwise is updated as: ( )

=

.kj ,

kj (v1)down ;

Update dual variables as per subsections ??- ?? ;

Update upper bound of subproblem D((v)) < Upperbound for (v) ;

end

U date LBD (v) if (v) > LBD

http://find/


36/47

Application to Process Synthesis Problems

Heat Exchanger Network Synthesis Problem

Cyclic Scheduling Problem

Feedtray location problemSynthesizing process flow sheet

Non-sharp separation in distillation

Multiproduct batch plant


http://find/


37/47

Heat exchanger network syntthesis I

Table: Heat exchanger network synthesis results

n |B| m p max nonl. var. conv.

Example-1 50 9 62 10 28 no

Example-2 50 9 62 10 28 no

Example-3 135 31 222 62 no

Pratik Patil



H h k h i II
http://find/


38/47

Heat exchanger network syntthesis II

GBD OA-AP LP/NLP B&B simplicial LagrangianDecomposition Decomposition

Example-1

Iterations 7 5 21 25 23of NLP-MIP

Total CPU 6.445 8.486 13.832 17.359 16.87time (sec.)

Annual cost 16062.49 15499.7 15499.7 167602 16235

Example-2

Iterations 9 5 27 30 22of NLP-MIP

Total CPU 7.482 9.76 15.91 18.548 17.356time (sec)

Annual cost 40296 37526 39759 44586 42653

Pratik Patil



H h k h i III
http://find/


39/47

Heat exchanger network syntthesis III

Example-3

GBD OA-AP GCD LP/NLP B & B

Iterations 11 8 10 30

of NLP-MIPTotal CPU 12.244 22.37 30.717 36.4611time (sec)

Annual cost 585618 601102 602043 586784

Pratik Patil



C li h d li bl I
http://find/


40/47

Cyclic scheduling problem I

Table: Cyclic scheduling results


38 12 20 5 21 noGBD OA-AP LP-NLP based B&B Simplicial Lagrangian SBB

Decompn. Decompn. (Global)

Iterations of 12 9 25 13 11NLP-MIP

Total CPU 12.49 18.14 30.12 20.81 11.16time (sec)

Total Profit 128833 128015 146821.38 136358.02 140104.6 153650

Pratik Patil



F dt l ti bl I
http://find/


41/47

Feedtray location problem I

Table: Feedtray location problem results


88 36 284 11 17 noGBD OA-ER-AP Simplicial Lagrangian SBBDecomposition Decomposition (Global)

Iterations of 10 7 13 10NLP-MIP

Reflux ratio (r) 0.9210 0.9899 0.9056 0.9103 0.99

Top product rate (P1) 66.134 69.57 65.367 64.42 69.61

Bottom product rate (P2) 33.86 30.41 34.63 35.58 30.38

Feedtray number 11 11 12 12 11

Total CPU 19.78 32.24 35.6 22.1536time (sec)

Pratik Patil



S th si i ss fl sh ts I
http://find/


42/47

Synthesizing process flow sheets I

Table: Synthesis of process system - Application of various methods


Prob.-1 7 3 6 3 2 yes

Prob.-2 2 5 14 5 2 yes

Prob.-3 18 8 23 8 2 yes

Pratik Patil



Synthesizing process flow sheets II
http://find/


43/47

Synthesizing process flow sheets II

Number of iterations OptimalGBD OA-AP LP-NLP B & B Simplicial Lagrangian Solution

Decomposition Decomposition

Prob.-1 5 4 4 8 6 6.01

Prob.-2 7 4 6 13 9 73.04

Prob.-3 9 7 9 20 16 68.01

Total CPU time (sec)GBD OA-AP LP-NLP B & B Simplicial Lagrangian


Prob.-1 0.49 0.7987 1.2838 0.882 0.5488

Prob.-2 1.77 2.8851 4.78 3.328 1.9834

Prob.-3 7.27 11.85 16.94 12,72 8.977

Pratik Patil



Non sharp separation in distillation
http://find/


44/47

Non-sharp separation in distillation

Table: Nonsharp separation results


51 2 43 15 5 noGBD Lagrangian Simplicial OA-AP


Iterations of 8 10 13 12LP-MIP


Objective:Total annual cost = 156700

with column I and II in series as starting point

Pratik Patil



Multiproduct Batch Plant
http://find/


45/47

Multiproduct Batch Plant

Table: Batch plant problems-results


Problem-1 20 9 20 5 2 no

Problem-2 47 24 73 10 2 noGBD OA LP/NLP B&B Lagrangian

Decomposition

Problem-1

Iterations of 5 3 10 7NLP-MIP


Optimal Investment cost: = 167427.66

Problem-2Pratik Patil


General Results
http://find/


46/47

CPU time Time taken by solver excluding compilation time

GBD < Block-separable techniques < OA-AP

Decomposition for optimzation.pdf

Documents