小崎敏寛 (Toshihiro Kosaki)

線形計画問題に対する主双対内点法における相補項の減少を考慮した

変数ごとのステップサイズの計算Computation of Indexwise Step Sizes Considering Reduction of Complementarity Terms in a Primal-Dual Interior-Point Method for Linear Problems

小崎敏寛 (Toshihiro Kosaki)

Spectral Method( in Optimization)スペクトル法

1. Introduction2. General Framework3. Interior-Point Method4. Applications5. Conclusions

RIMS2012 　 2012/07/24 2

1 Introduction( Spectral Method)

x*

x in R2

kkkk xxx 1

kk x11

kk x22

kk21 and are distinct.

Δxk

kx1

kx2

A (general) iterative algorithm for optimization problems

RIMS2012 　 2012/07/24 3

1 Spectral Method (cont.)

• R6

• Spectral Method(スペクトル法 )

Δx:direction

α:step size

RIMS2012 　 2012/07/24 4

2 General Framework

1. An initial point is given.2. If termination condition is satisfied then stop.3. Compute a direction .4. Compute step sizes .5. Update

.

6. Update k:=k+1.7. Go to 2.

),...,1(0 Nixi

),...,1( Niki

),...,1( Nixki

kN

kN

kk

kN

k

kN

k

x

x

x

x

x

x

111

1

11

RIMS2012 　 2012/07/24 5

2 General Framework (cont.)

Features1. An iterative algorithm with computing indexwise

step sizes ‘s. 2. x is in a finite dimension.3. x is a continuous variable.4. Can compute easily all appropriate indexwise

step sizes ‘s. (time of computing )

>>(time of computing )

kN

kN

kk

kN

k

kN

k

x

x

x

x

x

x

111

1

11

ki

kikix

kiRIMS2012 　 2012/07/24 6

3 (Primal-Dual) Interior-Point Method (for LP)

)conditionoptimality(0,0,,0,

)problemdual(0,..max

)problemprimal(0,..min

XzzczyAxbAx

zczyAtsyb

xbAxtsxc

T

TT

T

• From an initial positive point w0:=(x0,y0,z0) with x0 >0 and z0 >0, keeping positivity and using both primal and dual problems, the algorithm seeks a point satisfying optimality condition.

RIMS2012 　 2012/07/24 7

3 Interior-Point Method (cont.)

x i z i ↓0, i=1,…,n (optimality)

||r||↓0, (infeasibility)s.t. x>0 and z>0,

where r is residual .

RIMS2012 　 2012/07/24 8

.ryADAT

3 Interior-Point Method (cont.)

.and)1,0[where

,

,

,

),,(:directionNewton

n

zx

XzezXxZ

zyAczyA

AxbxA

zyxw

T

TT

central path

Δw

Most of computational cost is solving normal equations:

RIMS2012 　 2012/07/24 9

3 Interior-Point Method (cont.)

• (total computational time)=(number of iterations)×(time per iteration)Predictor-corrector methodMultiple-centering method

CG method

RIMS2012 　 2012/07/24 10

3 Proposition1

Proposition1:Relation between positivity and xizi

xi(α):=xi(0)+αΔxi, zi(α):=zi(0)+αΔzi (i=1,…,n).

Suppose xi(0)>0 and zi(0)>0 (i=1,…,n).

),...,1(0)()(

),...,1(0)(and0)(

),,0[],1,0[ **

nizx

nizx

ii

ii

RIMS2012 　 2012/07/24 11

3 Proposition2Proposition2:Relation between step size α and xTz

α10

xTz

γxTz

α2 Δ xTΔz

RIMS2012 　 2012/07/24 12

Neglecting quadratic term, reduction of xTz is in proportion to α.

3 Proposition3

Proposition3:Relation between step size α and residual r

1α

r1

r2

RIMS2012 　 2012/07/24 13

Reduction of r’s is in proportion to α.

3 Interior-Point Method (cont.)

• IPM has been extended from LP to QP, LCP, SOCP and SDP.

RIMS2012 　 2012/07/24 14

4 Applications

1. LP12. LP23. SDP4. SOCP

RIMS2012 　 2012/07/24 15

**

*

*

}{min4nPropositio

,...,1}}0:max{,1min{:

}}0:max{,1min{:

ii

iiiici

c

nixx

xx

4-1 LP1

e.g. αc=0.99995

Computation of Δy Computation of αi* ’s

Cholesky decompositionADAT

O(m3)O(m2n)

O(n)

RIMS2012 　 2012/07/24 16

4-1 LP1 (cont.)

• Numerical experiment

• Transportation problem• M: # of supply nodes, N: # of demand nodes• LP with M×N variables

RIMS2012 　 2012/07/24 17

IPM proposed IPM

M=N # of iterations time (sec.) # of iterations Time (sec.)

50 6 3.10×10-2 6 3.10×10-2

100 7 1.09×10-1 6 7.80×10-2

500 11 5.42 6 3.06

4-2 LP2• Minimizing

→Analytical solutions are available from quadratic equations

10,0)()(s.t.)()( iiiiiiiii zxzx xi(αi):=xi(0)+α i Δxi, zi(αi):=zi(0)+αiΔzi (i=1,…,n).

x i z i

0 1α i

•The smaller xi zi is, the better it is.•The larger αi is, the better it is.

ideal

RIMS2012 　 2012/07/24 18

4-3 SDP

Block diagonal SDP

RIMS2012 　 2012/07/24

OXXO

OXX

OXO

OX

222

111

2

1variable

19

4-4 SOCP

Direct product of second order conesvariables:

Δx1 Δx2

K1K2

x1 x2

×

RIMS2012 　 2012/07/24

.)(...)(

,)(...)(

22221

20

21211

10

2

1

n

n

xxx

xxx

20

5 Conclusions

• We proposed a general framework (Spectral Method).

• The framework was applied to some optimization problems.

RIMS2012 　 2012/07/24 21