Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables.

Easy Optimization Problems,

Relaxation,Local Processing

for a small subset of variables

Changes in the energy and overlap

X-direction line search and “discrete derivatives”

For node , fix all other nodes at their current Current overlap

------- ---------------- -------------------------

Calculate => choose sign Calculate sign => quadratic approx.

To effectively calculate the derivative which means:

Calculate and average:

j

jij

iji xxaxE 2)~()( ),( jii jx

~

)(,)(ii xixi xExE

)2(ixixE

in change unit per in change of rate the ii xxE )(

)(,)(ii xixi xExE

2

)/()]()([/)]()([iiii xixixixi xExExExE

Linesearch

Discretederivative

Different types of relaxation

Variable by variable relaxation – strict minimization

Changing a small subset of variables simultaneously – Window strict minimization relaxation

Stochastic relaxation – may increase the energy – should be followed by strict minimization

Window strict unconstrained minimization

Discrete (combinatorial) case :

Permutations of small subsets P=2, placement

1D data base

The nodes: 1 2 3 4 5 6 7 8 9A permutation 5 39 6 2 7 1 4 8 (1)= 7 , (2)=5 , (3)=2 …

To find a consecutive subset of nodes in the current permutation, we need the inverse of -1

-1(1)= 5 , -1(2)= 3 , -1(3)= 9 …

In 2D we have to insert a grid and store the list of nodes within each square

Window strict unconstrained minimization

Discrete (combinatorial) case :

Permutations of small subsets P=2, placement

Problem: very small number of variables! Quadratic case : P=2

Window relaxation for P=2 unconstrained version

Minimize

Pick a window of variables , fix all variables at Find a correctionto so as to

minimize

Quadratic functional in many variables – easy to solve!

ij

jiij xxax 2)()(

2

,,

2 ~~~~)( )xx(a)xx(a jiijjiij iWjWiWji

ji

bAi

systemlinear the solve0)(

)(

Wi x~

i Wixi ,~

Updating the window variables For each i in the window W

insert the correction: xi new

=xi+i

Sort the xi news and rearrange the window

accordingly To improve the result obtained by the

inner changes apply node-by-node relaxation on W and on its boundary

At the and compare the “old” energy with the “new” energy and accept / reject

Revision process: try a “big” change, improve it by local minimization, choose

Window relaxation for P=2 constrained version

To prevent nodes from collapsing on each other To express the aim of having an

approximate permutation of add 2 constraints:

Wiiix }~{

2,1,~)~(

mvxvx i

m

Wiii

mi

Wii

Wiix }~{

Exc#4: Permutation’s invariants

1) Prove that are

invariant under permutation.

2) Is it also true for m=3?

2,1,~1

mvx i

n

i

mi

Window relaxation for P=2 constrained version

To prevent nodes from collapsing on each other To express the aim of having an

approximate permutation of add 2 constraints:

Minimization with equality constraints Lagrange multipliers

Wiiix }~{

2,1,~)~(

mvxvx i

m

Wiii

mi

Wii

Wiix }~{

0~2

01

Wiiii

Wiii

xvm

vm

Lagrange multipliers

Goal: Transform a constrained optimization problem with n variables and m equality constraints to an unconstrained optimization problem with n+m variables. The new m variables are called the Lagrange multipliers

Geometry explanation

2 constraints in 3D

The optimal ellipsoid is tangent to the constraints curve

Lagrange multipliers

Goal: Transform a constrained optimization problem with n variables and m equality constraints to an unconstrained optimization problem with n+m variables. The new m variables are called the Lagrange multipliers

Geometry explanation Construct an augmented functional –

the Lagrangian

The LagrangianGiven E(x) subject to m equality constraints:

hk(x)=0 , k=1,…,m , construct the Lagrangian

L(x,) = E(x) + kkhk(x) and solve the system

of n+m equations

The value of is meaningful

mkxL

nix

xL

k

i

,...,1,0),(

,...,1,0),(

for

for

The constraints!

The Lagrangian: an example Minimize E(x,y)=x+y Subject to h(x,y)=x2+y2-2 The Lagrangian: L(x,y,=E(x,y)+(x2+y2-2

020

0210

0210

0

22

yxE

yy

E

xx

E

hEhEL

The constraint!

The co-linearityof the gradients

Window relaxation for 1D ordering constrained/unconstrained version

Minimize

Pick a window of variables , fix all variables at Find a correctionto Update the window’s variables, restore volume

constraints and revise around the window

Switch to the next window chosen with overlap Use a (small) sequence of variable size windows For example use windows with 5,10,15,20,25 nodes

ij

Pjiij xxax ||)(

Wi x~

i Wixi ,~

Easy to solve problems Quadratic functional and linear constraintsSolve a linear system of equationsQuadratization of the functional: P=1, P>2

Quadratization for P=1 and P>2

Minimize ;

Given a current approximation

Minimize

Minimize

ij

jiij xxax ||)(

ij

ji

ji

ij

xxxx

ax 2)(

|~~|)(ˆ

ix~

ij

jiij xxax 6)()(

ij

ji

ji

ij

xxxx

ax 2

4)(

)~~()(ˆ

Easy to solve problems Quadratic functional and linear constraintsSolve a linear system of equationsQuadratization of the functional: P=1, P>2Linearization of the constraints: P=2

Window relaxation for P=2 constrained version

To prevent nodes from collapsing on each other To express the aim of having an

approximate permutation of add 2 constraints:

The terms were neglected assuming they are small enough compared with other terms in the equation

Wiiix }~{

2,1,~)~(

mvxvx i

m

Wiii

mi

Wii

Wiix }~{

0~2

01

Wiiii

Wiii

xvm

vm

2

Easy to solve problems Quadratic functional and linear constraintsSolve a linear system of equationsQuadratization of the functional: P=1, P>2Linearization of the constraints: P=2

Inequality constraints: active set method