Top Banner
Bounded degree SOS hierarchy for polynomial optimization Jean B. Lasserre LAAS-CNRS and Institute of Mathematics, Toulouse, France Joint work with K. Toh and S. Yang (NUS, Singapore) MINLP workshop, Sevilla, April 2015 Jean B. Lasserre semidefinite characterization
78

Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Jul 18, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Bounded degree SOS hierarchy for polynomialoptimization

Jean B. Lasserre

LAAS-CNRS and Institute of Mathematics, Toulouse, France

Joint work with K. Toh and S. Yang (NUS, Singapore)

MINLP workshop, Sevilla, April 2015

Jean B. Lasserre semidefinite characterization

Page 2: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Why polynomial optimization?LP- and SDP- CERTIFICATES of POSITIVITYThe moment-LP and moment-SOS approachesBounded degree SOS hierarchy

Jean B. Lasserre semidefinite characterization

Page 3: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Why polynomial optimization?LP- and SDP- CERTIFICATES of POSITIVITYThe moment-LP and moment-SOS approachesBounded degree SOS hierarchy

Jean B. Lasserre semidefinite characterization

Page 4: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Why polynomial optimization?LP- and SDP- CERTIFICATES of POSITIVITYThe moment-LP and moment-SOS approachesBounded degree SOS hierarchy

Jean B. Lasserre semidefinite characterization

Page 5: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Why polynomial optimization?LP- and SDP- CERTIFICATES of POSITIVITYThe moment-LP and moment-SOS approachesBounded degree SOS hierarchy

Jean B. Lasserre semidefinite characterization

Page 6: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

With f ∈ R[x] and K := {x ∈ Rn : gj(x) ≥ 0, j = 1, . . . ,m } acompact basic semi-algebraic set (i.e., the gj ’s are alsopolynomials),

consider the global polynomial optimization problem:

f ∗ = minx{ f (x) : x ∈ K }

Remember that for the GLOBAL minimum f ∗:

f ∗ = sup {λ : f (x)− λ ≥ 0 ∀x ∈ K}.

... and so to compute f ∗ one needs

TRACTABLE CERTIFICATES of POSITIVITY on K!

Jean B. Lasserre semidefinite characterization

Page 7: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

With f ∈ R[x] and K := {x ∈ Rn : gj(x) ≥ 0, j = 1, . . . ,m } acompact basic semi-algebraic set (i.e., the gj ’s are alsopolynomials),

consider the global polynomial optimization problem:

f ∗ = minx{ f (x) : x ∈ K }

Remember that for the GLOBAL minimum f ∗:

f ∗ = sup {λ : f (x)− λ ≥ 0 ∀x ∈ K}.

... and so to compute f ∗ one needs

TRACTABLE CERTIFICATES of POSITIVITY on K!

Jean B. Lasserre semidefinite characterization

Page 8: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

With f ∈ R[x] and K := {x ∈ Rn : gj(x) ≥ 0, j = 1, . . . ,m } acompact basic semi-algebraic set (i.e., the gj ’s are alsopolynomials),

consider the global polynomial optimization problem:

f ∗ = minx{ f (x) : x ∈ K }

Remember that for the GLOBAL minimum f ∗:

f ∗ = sup {λ : f (x)− λ ≥ 0 ∀x ∈ K}.

... and so to compute f ∗ one needs

TRACTABLE CERTIFICATES of POSITIVITY on K!

Jean B. Lasserre semidefinite characterization

Page 9: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

REAL ALGEBRAIC GEOMETRY helps!!!!

Indeed, POWERFUL CERTIFICATES OF POSITIVITY EXIST!

Moreover .... and importantly,

Such certificates are amenable to PRACTICAL COMPUTATION!

(? Stronger Positivstellensatzë exist for analytic functions butare useless from a computational viewpoint.)

Jean B. Lasserre semidefinite characterization

Page 10: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

REAL ALGEBRAIC GEOMETRY helps!!!!

Indeed, POWERFUL CERTIFICATES OF POSITIVITY EXIST!

Moreover .... and importantly,

Such certificates are amenable to PRACTICAL COMPUTATION!

(? Stronger Positivstellensatzë exist for analytic functions butare useless from a computational viewpoint.)

Jean B. Lasserre semidefinite characterization

Page 11: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

REAL ALGEBRAIC GEOMETRY helps!!!!

Indeed, POWERFUL CERTIFICATES OF POSITIVITY EXIST!

Moreover .... and importantly,

Such certificates are amenable to PRACTICAL COMPUTATION!

(? Stronger Positivstellensatzë exist for analytic functions butare useless from a computational viewpoint.)

Jean B. Lasserre semidefinite characterization

Page 12: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

SOS-based certificate

K = {x : gj(x) ≥ 0, j = 1, . . . ,m}

Theorem (Putinar’s Positivstellensatz)If K is compact (+ a technical Archimedean assumption) andf > 0 on K then:

† f (x) = σ0(x) +m∑

j=1

σj(x)gj(x), ∀x ∈ Rn,

for some SOS polynomials (σj) ⊂ R[x].

Testing whether † holds for some

SOS (σj) ⊂ R[x] with a degree bound, is SOLVING an SDP!

Jean B. Lasserre semidefinite characterization

Page 13: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

SOS-based certificate

K = {x : gj(x) ≥ 0, j = 1, . . . ,m}

Theorem (Putinar’s Positivstellensatz)If K is compact (+ a technical Archimedean assumption) andf > 0 on K then:

† f (x) = σ0(x) +m∑

j=1

σj(x)gj(x), ∀x ∈ Rn,

for some SOS polynomials (σj) ⊂ R[x].

Testing whether † holds for some

SOS (σj) ⊂ R[x] with a degree bound, is SOLVING an SDP!

Jean B. Lasserre semidefinite characterization

Page 14: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

LP-based certificate

K = {x : gj(x) ≥ 0; (1− gj(x)) ≥ 0, j = 1, . . . ,m}

Theorem (Krivine-Vasilescu-Handelman’s Positivstellensatz)

Let K be compact and the family {1,gj} generate R[x]. If f > 0on K then:

? f (x) =∑α,β

cαβm∏

j=1

gj(x)αj (1− gj(x))βj , , ∀x ∈ Rn,

for some NONNEGATIVE scalars (cαβ).

Testing whether ? holds for some

NONNEGATIVE (cαβ) with |α+ β| ≤ M, is SOLVING an LP!

Jean B. Lasserre semidefinite characterization

Page 15: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

LP-based certificate

K = {x : gj(x) ≥ 0; (1− gj(x)) ≥ 0, j = 1, . . . ,m}

Theorem (Krivine-Vasilescu-Handelman’s Positivstellensatz)

Let K be compact and the family {1,gj} generate R[x]. If f > 0on K then:

? f (x) =∑α,β

cαβm∏

j=1

gj(x)αj (1− gj(x))βj , , ∀x ∈ Rn,

for some NONNEGATIVE scalars (cαβ).

Testing whether ? holds for some

NONNEGATIVE (cαβ) with |α+ β| ≤ M, is SOLVING an LP!

Jean B. Lasserre semidefinite characterization

Page 16: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

SUCH POSITIVITY CERTIFICATES

allow to infer GLOBAL Properties ofFEASIBILITY and OPTIMALITY,

... the analogue of (well-known) previous ones

valid in the CONVEX CASE ONLY!

Farkas Lemma→ Krivine-StengleKKT-Optimality conditions→ Schmüdgen-Putinar

Jean B. Lasserre semidefinite characterization

Page 17: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

SUCH POSITIVITY CERTIFICATES

allow to infer GLOBAL Properties ofFEASIBILITY and OPTIMALITY,

... the analogue of (well-known) previous ones

valid in the CONVEX CASE ONLY!

Farkas Lemma→ Krivine-StengleKKT-Optimality conditions→ Schmüdgen-Putinar

Jean B. Lasserre semidefinite characterization

Page 18: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

SUCH POSITIVITY CERTIFICATES

allow to infer GLOBAL Properties ofFEASIBILITY and OPTIMALITY,

... the analogue of (well-known) previous ones

valid in the CONVEX CASE ONLY!

Farkas Lemma→ Krivine-StengleKKT-Optimality conditions→ Schmüdgen-Putinar

Jean B. Lasserre semidefinite characterization

Page 19: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

SUCH POSITIVITY CERTIFICATES

allow to infer GLOBAL Properties ofFEASIBILITY and OPTIMALITY,

... the analogue of (well-known) previous ones

valid in the CONVEX CASE ONLY!

Farkas Lemma→ Krivine-StengleKKT-Optimality conditions→ Schmüdgen-Putinar

Jean B. Lasserre semidefinite characterization

Page 20: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

• In addition, polynomials NONNEGATIVE ON A SET K ⊂ Rn

are ubiquitous. They also appear in many importantapplications (outside optimization), . . . modeled as

particular instances of the so calledGeneralized Moment Problem, among which:

Probability, Optimal and Robust Control, Game theory, Signalprocessing, multivariate integration, etc.

Jean B. Lasserre semidefinite characterization

Page 21: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

• In addition, polynomials NONNEGATIVE ON A SET K ⊂ Rn

are ubiquitous. They also appear in many importantapplications (outside optimization), . . . modeled as

particular instances of the so calledGeneralized Moment Problem, among which:

Probability, Optimal and Robust Control, Game theory, Signalprocessing, multivariate integration, etc.

Jean B. Lasserre semidefinite characterization

Page 22: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

For instance, one may also want:• To approximate sets defined with QUANTIFIERS, like .e.g.,

Rf := {x ∈ B : f (x , y) ≤ 0 for all y such that (x , y) ∈ K}

Df := {x ∈ B : f (x , y) ≤ 0 for some y such that (x , y) ∈ K}

where f ∈ R[x , y ], B is a simple set (box, ellipsoid).

• To compute convex polynomial underestimators p ≤ f of apolynomial f on a box B ⊂ Rn. (Very useful in MINLP.)

Jean B. Lasserre semidefinite characterization

Page 23: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

For instance, one may also want:• To approximate sets defined with QUANTIFIERS, like .e.g.,

Rf := {x ∈ B : f (x , y) ≤ 0 for all y such that (x , y) ∈ K}

Df := {x ∈ B : f (x , y) ≤ 0 for some y such that (x , y) ∈ K}

where f ∈ R[x , y ], B is a simple set (box, ellipsoid).

• To compute convex polynomial underestimators p ≤ f of apolynomial f on a box B ⊂ Rn. (Very useful in MINLP.)

Jean B. Lasserre semidefinite characterization

Page 24: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

The moment-LP and moment-SOS approachesconsist of using a certain type of positivity certificate(Krivine-Vasilescu-Handelman’s or Putinar’s certificate) inpotentially any application where such a characterization isneeded. (Global optimization is only one example.)

In many situations this amounts tosolving a HIERARCHY of :

LINEAR PROGRAMS, orSEMIDEFINITE PROGRAMS

... of increasing size!.

Jean B. Lasserre semidefinite characterization

Page 25: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

The moment-LP and moment-SOS approachesconsist of using a certain type of positivity certificate(Krivine-Vasilescu-Handelman’s or Putinar’s certificate) inpotentially any application where such a characterization isneeded. (Global optimization is only one example.)

In many situations this amounts tosolving a HIERARCHY of :

LINEAR PROGRAMS, orSEMIDEFINITE PROGRAMS

... of increasing size!.

Jean B. Lasserre semidefinite characterization

Page 26: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

The moment-LP and moment-SOS approachesconsist of using a certain type of positivity certificate(Krivine-Vasilescu-Handelman’s or Putinar’s certificate) inpotentially any application where such a characterization isneeded. (Global optimization is only one example.)

In many situations this amounts tosolving a HIERARCHY of :

LINEAR PROGRAMS, orSEMIDEFINITE PROGRAMS

... of increasing size!.

Jean B. Lasserre semidefinite characterization

Page 27: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

The moment-LP and moment-SOS approachesconsist of using a certain type of positivity certificate(Krivine-Vasilescu-Handelman’s or Putinar’s certificate) inpotentially any application where such a characterization isneeded. (Global optimization is only one example.)

In many situations this amounts tosolving a HIERARCHY of :

LINEAR PROGRAMS, orSEMIDEFINITE PROGRAMS

... of increasing size!.

Jean B. Lasserre semidefinite characterization

Page 28: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

LP- and SDP-hierarchies for optimization

Replace f ∗ = supλ,σj{λ : f (x)− λ ≥ 0 ∀x ∈ K} with:

The SDP-hierarchy indexed by d ∈ N:

f ∗d = sup {λ : f − λ = σ0︸︷︷︸SOS

+m∑

j=1

σj︸︷︷︸SOS

gj ; deg (σj gj) ≤ 2d }

or, the LP-hierarchy indexed by d ∈ N:

θd = sup {λ : f −λ =∑α,β

cαβ︸︷︷︸≥0

m∏j=1

gjαj (1−gj)

βj ; |α+β| ≤ 2d}

Jean B. Lasserre semidefinite characterization

Page 29: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

LP- and SDP-hierarchies for optimization

Replace f ∗ = supλ,σj{λ : f (x)− λ ≥ 0 ∀x ∈ K} with:

The SDP-hierarchy indexed by d ∈ N:

f ∗d = sup {λ : f − λ = σ0︸︷︷︸SOS

+m∑

j=1

σj︸︷︷︸SOS

gj ; deg (σj gj) ≤ 2d }

or, the LP-hierarchy indexed by d ∈ N:

θd = sup {λ : f −λ =∑α,β

cαβ︸︷︷︸≥0

m∏j=1

gjαj (1−gj)

βj ; |α+β| ≤ 2d}

Jean B. Lasserre semidefinite characterization

Page 30: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

TheoremBoth sequence (f ∗d ), and (θd), d ∈ N, are MONOTONE NONDECREASING and when K is compact (and satisfies atechnical Archimedean assumption) then:

f ∗ = limd→∞

f ∗d = limd→∞

θd .

Jean B. Lasserre semidefinite characterization

Page 31: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

•What makes this approach exciting is that it is at thecrossroads of several disciplines/applications:

Commutative, Non-commutative, and Non-linearALGEBRAReal algebraic geometry, and Functional AnalysisOptimization, Convex AnalysisComputational Complexity in Computer Science,

which BENEFIT from interactions!

• As mentioned ... potential applications are ENDLESS!

Jean B. Lasserre semidefinite characterization

Page 32: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

•What makes this approach exciting is that it is at thecrossroads of several disciplines/applications:

Commutative, Non-commutative, and Non-linearALGEBRAReal algebraic geometry, and Functional AnalysisOptimization, Convex AnalysisComputational Complexity in Computer Science,

which BENEFIT from interactions!

• As mentioned ... potential applications are ENDLESS!

Jean B. Lasserre semidefinite characterization

Page 33: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

• Has already been proved useful and successful inapplications with modest problem size, notably in optimization,control, robust control, optimal control, estimation, computervision, etc. (If sparsity then problems of larger size can beaddressed)

• HAS initiated and stimulated new research issues:in Convex Algebraic Geometry (e.g. semidefiniterepresentation of convex sets, algebraic degree ofsemidefinite programming and polynomial optimization)in Computational algebra (e.g., for solving polynomialequations via SDP and Border bases)Computational Complexity where LP- andSDP-HIERARCHIES have become an important tool toanalyze Hardness of Approximation for 0/1 combinatorialproblems (→ links with quantum computing)

Jean B. Lasserre semidefinite characterization

Page 34: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

• Has already been proved useful and successful inapplications with modest problem size, notably in optimization,control, robust control, optimal control, estimation, computervision, etc. (If sparsity then problems of larger size can beaddressed)

• HAS initiated and stimulated new research issues:in Convex Algebraic Geometry (e.g. semidefiniterepresentation of convex sets, algebraic degree ofsemidefinite programming and polynomial optimization)in Computational algebra (e.g., for solving polynomialequations via SDP and Border bases)Computational Complexity where LP- andSDP-HIERARCHIES have become an important tool toanalyze Hardness of Approximation for 0/1 combinatorialproblems (→ links with quantum computing)

Jean B. Lasserre semidefinite characterization

Page 35: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Recall that both LP- and SDP- hierarchies areGENERAL PURPOSE METHODS ....

NOT TAILORED to solving specific hard problems!!

Jean B. Lasserre semidefinite characterization

Page 36: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Recall that both LP- and SDP- hierarchies areGENERAL PURPOSE METHODS ....

NOT TAILORED to solving specific hard problems!!

Jean B. Lasserre semidefinite characterization

Page 37: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

A remarkable property of the SOS hierarchy: I

When solving the optimization problem

P : f ∗ = min {f (x) : gj(x) ≥ 0, j = 1, . . . ,m}

one does NOT distinguish between CONVEX, CONTINUOUSNON CONVEX, and 0/1 (and DISCRETE) problems! A booleanvariable xi is modelled via the equality constraint “x2

i − xi = 0".

In Non Linear Programming (NLP),

modeling a 0/1 variable with the polynomial equality constraint“x2

i − xi = 0"and applying a standard descent algorithm would be

considered “stupid"!

Each class of problems has its own ad hoc tailored algorithms.

Jean B. Lasserre semidefinite characterization

Page 38: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

A remarkable property of the SOS hierarchy: I

When solving the optimization problem

P : f ∗ = min {f (x) : gj(x) ≥ 0, j = 1, . . . ,m}

one does NOT distinguish between CONVEX, CONTINUOUSNON CONVEX, and 0/1 (and DISCRETE) problems! A booleanvariable xi is modelled via the equality constraint “x2

i − xi = 0".

In Non Linear Programming (NLP),

modeling a 0/1 variable with the polynomial equality constraint“x2

i − xi = 0"and applying a standard descent algorithm would be

considered “stupid"!

Each class of problems has its own ad hoc tailored algorithms.

Jean B. Lasserre semidefinite characterization

Page 39: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

A remarkable property of the SOS hierarchy: I

When solving the optimization problem

P : f ∗ = min {f (x) : gj(x) ≥ 0, j = 1, . . . ,m}

one does NOT distinguish between CONVEX, CONTINUOUSNON CONVEX, and 0/1 (and DISCRETE) problems! A booleanvariable xi is modelled via the equality constraint “x2

i − xi = 0".

In Non Linear Programming (NLP),

modeling a 0/1 variable with the polynomial equality constraint“x2

i − xi = 0"and applying a standard descent algorithm would be

considered “stupid"!

Each class of problems has its own ad hoc tailored algorithms.

Jean B. Lasserre semidefinite characterization

Page 40: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Even though the moment-SOS approach DOES NOTSPECIALIZE to each class of problems:

It recognizes the class of (easy) SOS-convex problems asFINITE CONVERGENCE occurs at the FIRST relaxation inthe hierarchy.Finite convergence also occurs for general convexproblems and generically for non convex problems→ (NOT true for the LP-hierarchy.)The SOS-hierarchy dominates other lift-and-projecthierarchies (i.e. provides the best lower bounds) for hard0/1 combinatorial optimization problems!

Jean B. Lasserre semidefinite characterization

Page 41: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Even though the moment-SOS approach DOES NOTSPECIALIZE to each class of problems:

It recognizes the class of (easy) SOS-convex problems asFINITE CONVERGENCE occurs at the FIRST relaxation inthe hierarchy.Finite convergence also occurs for general convexproblems and generically for non convex problems→ (NOT true for the LP-hierarchy.)The SOS-hierarchy dominates other lift-and-projecthierarchies (i.e. provides the best lower bounds) for hard0/1 combinatorial optimization problems!

Jean B. Lasserre semidefinite characterization

Page 42: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Even though the moment-SOS approach DOES NOTSPECIALIZE to each class of problems:

It recognizes the class of (easy) SOS-convex problems asFINITE CONVERGENCE occurs at the FIRST relaxation inthe hierarchy.Finite convergence also occurs for general convexproblems and generically for non convex problems→ (NOT true for the LP-hierarchy.)The SOS-hierarchy dominates other lift-and-projecthierarchies (i.e. provides the best lower bounds) for hard0/1 combinatorial optimization problems!

Jean B. Lasserre semidefinite characterization

Page 43: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Even though the moment-SOS approach DOES NOTSPECIALIZE to each class of problems:

It recognizes the class of (easy) SOS-convex problems asFINITE CONVERGENCE occurs at the FIRST relaxation inthe hierarchy.Finite convergence also occurs for general convexproblems and generically for non convex problems→ (NOT true for the LP-hierarchy.)The SOS-hierarchy dominates other lift-and-projecthierarchies (i.e. provides the best lower bounds) for hard0/1 combinatorial optimization problems!

Jean B. Lasserre semidefinite characterization

Page 44: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Even though the moment-SOS approach DOES NOTSPECIALIZE to each class of problems:

It recognizes the class of (easy) SOS-convex problems asFINITE CONVERGENCE occurs at the FIRST relaxation inthe hierarchy.Finite convergence also occurs for general convexproblems and generically for non convex problems→ (NOT true for the LP-hierarchy.)The SOS-hierarchy dominates other lift-and-projecthierarchies (i.e. provides the best lower bounds) for hard0/1 combinatorial optimization problems!

Jean B. Lasserre semidefinite characterization

Page 45: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

A remarkable property: II

FINITE CONVERGENCE of the SOS-hierarchy is GENERIC!

... and provides a GLOBAL OPTIMALITY CERTIFICATE,

the analogue for the NON CONVEX CASE of the

KKT-OPTIMALITY conditions in the CONVEX CASE!

Jean B. Lasserre semidefinite characterization

Page 46: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Theorem (Marshall, Nie)Let x∗ ∈ K be a global minimizer of

P : f ∗ = min {f (x) : gj(x) ≥ 0, j = 1, . . . ,m}.

and assume that:(i) The gradients {∇gj(x∗)} are linearly independent,(ii) Strict complementarity holds (λ∗j gj(x∗) = 0 for all j .)(iii) Second-order sufficiency conditions hold at

(x∗, λ∗) ∈ K× Rm+.

Then f (x)− f ∗ = σ∗0(x) +m∑

j=1

σ∗j (x)gj(x), ∀x ∈ Rn, for some

SOS polynomials {σ∗j }.

Moreover, the conditions (i)-(ii)-(iii) HOLD GENERICALLY!

Jean B. Lasserre semidefinite characterization

Page 47: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Theorem (Marshall, Nie)Let x∗ ∈ K be a global minimizer of

P : f ∗ = min {f (x) : gj(x) ≥ 0, j = 1, . . . ,m}.

and assume that:(i) The gradients {∇gj(x∗)} are linearly independent,(ii) Strict complementarity holds (λ∗j gj(x∗) = 0 for all j .)(iii) Second-order sufficiency conditions hold at

(x∗, λ∗) ∈ K× Rm+.

Then f (x)− f ∗ = σ∗0(x) +m∑

j=1

σ∗j (x)gj(x), ∀x ∈ Rn, for some

SOS polynomials {σ∗j }.

Moreover, the conditions (i)-(ii)-(iii) HOLD GENERICALLY!

Jean B. Lasserre semidefinite characterization

Page 48: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Certificates of positivity already exist in convex optimization

f ∗ = f (x∗) = min { f (x) : gj(x) ≥ 0, j = 1, . . . ,m }

when f and −gj are CONVEX. Indeed if Slater’s condition holdsthere exist nonnegative KKT-multipliers λ∗j ∈ Rm

+ such that:

∇f (x∗)−m∑

j=1

λj∗ gj(x∗) = 0; λj

∗ gj(x∗) = 0, j = 1, . . . ,m.

... and so ... the Lagrangian

Lλ∗(x) := f (x)− f ∗ −∑j=1

λj∗ gj(x),

satisfiesLλ∗(x∗) = 0 and Lλ∗(x) ≥ 0 for all x. Therefore:

Lλ∗(x) ≥ 0⇒ f (x) ≥ f ∗ ∀x ∈ K!

Jean B. Lasserre semidefinite characterization

Page 49: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Certificates of positivity already exist in convex optimization

f ∗ = f (x∗) = min { f (x) : gj(x) ≥ 0, j = 1, . . . ,m }

when f and −gj are CONVEX. Indeed if Slater’s condition holdsthere exist nonnegative KKT-multipliers λ∗j ∈ Rm

+ such that:

∇f (x∗)−m∑

j=1

λj∗ gj(x∗) = 0; λj

∗ gj(x∗) = 0, j = 1, . . . ,m.

... and so ... the Lagrangian

Lλ∗(x) := f (x)− f ∗ −∑j=1

λj∗ gj(x),

satisfiesLλ∗(x∗) = 0 and Lλ∗(x) ≥ 0 for all x. Therefore:

Lλ∗(x) ≥ 0⇒ f (x) ≥ f ∗ ∀x ∈ K!

Jean B. Lasserre semidefinite characterization

Page 50: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

In summary:

KKT-OPTIMALITY PUTINAR’s CERTIFICATEwhen f and −gj are CONVEX in the non CONVEX CASE

∇f (x∗)−m∑

j=1

λ∗j ∇gj(x∗) = 0 ∇f (x∗)−m∑

j=1

σj(x∗)∇gj(x∗) = 0

f (x)− f ∗ −m∑

j=1

λ∗j gj(x) f (x)− f ∗ −m∑

j=1

σ∗j (x)gj(x)

≥ 0 for all x ∈ Rn (= σ∗0(x)) ≥ 0 for all x ∈ Rn.

for some SOS {σ∗j }, andσ∗j (x

∗) = λ∗j .

Jean B. Lasserre semidefinite characterization

Page 51: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

In summary:

KKT-OPTIMALITY PUTINAR’s CERTIFICATEwhen f and −gj are CONVEX in the non CONVEX CASE

∇f (x∗)−m∑

j=1

λ∗j ∇gj(x∗) = 0 ∇f (x∗)−m∑

j=1

σj(x∗)∇gj(x∗) = 0

f (x)− f ∗ −m∑

j=1

λ∗j gj(x) f (x)− f ∗ −m∑

j=1

σ∗j (x)gj(x)

≥ 0 for all x ∈ Rn (= σ∗0(x)) ≥ 0 for all x ∈ Rn.

for some SOS {σ∗j }, andσ∗j (x

∗) = λ∗j .

Jean B. Lasserre semidefinite characterization

Page 52: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

So even though both LP- and SDP-relaxations were notdesigned for solving specific hard problems ...

The SDP-relaxations behave reasonably well ("efficiently"?) invery different contexts in contrast to LP-relaxations.

However they also have limits to their efficiency mainly becauseof severe size limit inherent to all SDP-solvers ...

Question: Could we define an alternative hierarchy whichcombines some of the advantages of the SOS-hierarchy ...WITHOUT ITS SEVERE SIZE LIMITATION?

Jean B. Lasserre semidefinite characterization

Page 53: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

So even though both LP- and SDP-relaxations were notdesigned for solving specific hard problems ...

The SDP-relaxations behave reasonably well ("efficiently"?) invery different contexts in contrast to LP-relaxations.

However they also have limits to their efficiency mainly becauseof severe size limit inherent to all SDP-solvers ...

Question: Could we define an alternative hierarchy whichcombines some of the advantages of the SOS-hierarchy ...WITHOUT ITS SEVERE SIZE LIMITATION?

Jean B. Lasserre semidefinite characterization

Page 54: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

A Lagrangian interpretation of LP-relaxations

Consider the optimization problem

P : f ∗ = min {f (x) : x ∈ K },

where K is the compact basic semi-algebraic set:

K := {x ∈ Rn : gj(x) ≥ 0; j = 1, . . . ,m }.

Assume that:

• For every j = 1, . . . ,m (and possibly after scaling), gj(x) ≤ 1for all x ∈ K.

• The family {1,gj} generate R[x].

Jean B. Lasserre semidefinite characterization

Page 55: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Lagrangian relaxation

The dual method of multipliers, or Lagrangian relaxationconsists of solving: ρ := maxu{G(u) : u ≥ 0 },

with u 7→ G(u) := minx

f (x)−m∑

j=1

uj gj(x)

.

Equivalently:

ρ = maxu,λ{λ : f (x)−

m∑j=1

uj gj(x) ≥ λ, ∀x .}

In general, there is a DUALITY GAP, i.e., ρ < f ∗,

except in the CONVEX case where f and −gj are all convex(and under some conditions).

Jean B. Lasserre semidefinite characterization

Page 56: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

With d ∈ N fixed, consider the new optimization problem Pd :

f ∗d = minx{ f (x) :

m∏j=1

gj(x)αj (1− gj(x))βj ≥ 0

∀α, β : |α+ β| =∑

j αj + βj ≤ 2d}

Of course

P and Pd are equivalent and so f ∗d = f ∗.

... because Pd is just P with additional redundant constraints!

Jean B. Lasserre semidefinite characterization

Page 57: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

The Lagrangian relaxation of Pd consists of solving:

ρd = maxu≥0,λ

{λ : f (x)−∑α,β

uαβm∏

j=1

gj(x)αj (1− gj(x))βj ≥ λ, ∀x .

|α+ β| ≤ 2d}

Theoremρd ≤ f ∗ for all d ∈ N. Moreover, if K is compact, the family ofpolynomials {1,gj} generates R[x], and 0 ≤ gj ≤ 1 on K for allj = 1, . . . ,m, then:

limd→∞

ρd = f ∗.

Jean B. Lasserre semidefinite characterization

Page 58: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

The Lagrangian relaxation of Pd consists of solving:

ρd = maxu≥0,λ

{λ : f (x)−∑α,β

uαβm∏

j=1

gj(x)αj (1− gj(x))βj ≥ λ, ∀x .

|α+ β| ≤ 2d}

Theoremρd ≤ f ∗ for all d ∈ N. Moreover, if K is compact, the family ofpolynomials {1,gj} generates R[x], and 0 ≤ gj ≤ 1 on K for allj = 1, . . . ,m, then:

limd→∞

ρd = f ∗.

Jean B. Lasserre semidefinite characterization

Page 59: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

The previous theorem provides a rationale

for the well-known fact that :

adding redundant constraints to P helps when doingrelaxations!

On the other hand ...

we don’t know HOW TO COMPUTE ρd !

Jean B. Lasserre semidefinite characterization

Page 60: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

The previous theorem provides a rationale

for the well-known fact that :

adding redundant constraints to P helps when doingrelaxations!

On the other hand ...

we don’t know HOW TO COMPUTE ρd !

Jean B. Lasserre semidefinite characterization

Page 61: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

The LP-hierarchy may be viewed as

the BRUTE FORCE SIMPLIFICATION of

ρd = maxu≥0,λ

{λ : f −∑α,β

uαβm∏

j=1

gjαj (1− gj)

βj − λ ≥ 0, on Rn

|α+ β| ≤ 2d}

to ...

θd = maxu≥0,λ

{λ : f −∑α,β

uαβm∏

j=1

gjαj (1− gj)

βj − λ = 0, on Rn

|α+ β| ≤ 2d} !!

Jean B. Lasserre semidefinite characterization

Page 62: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

The LP-hierarchy may be viewed as

the BRUTE FORCE SIMPLIFICATION of

ρd = maxu≥0,λ

{λ : f −∑α,β

uαβm∏

j=1

gjαj (1− gj)

βj − λ ≥ 0, on Rn

|α+ β| ≤ 2d}

to ...

θd = maxu≥0,λ

{λ : f −∑α,β

uαβm∏

j=1

gjαj (1− gj)

βj − λ = 0, on Rn

|α+ β| ≤ 2d} !!

Jean B. Lasserre semidefinite characterization

Page 63: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

and indeed, ... with |α+ β| ≤ 2d ,

the set of (u, λ) such that u ≥ 0 and

f (x)−∑α,β

uαβm∏

j=1

gj(x)αj (1− gj(x))βj − λ = 0, ∀x .

is a CONVEX POLYTOPE!

and so, computing θd is solving a Linear Program!

and one has f ∗ ≥ ρd ≥ θd for all d .

Jean B. Lasserre semidefinite characterization

Page 64: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

and indeed, ... with |α+ β| ≤ 2d ,

the set of (u, λ) such that u ≥ 0 and

f (x)−∑α,β

uαβm∏

j=1

gj(x)αj (1− gj(x))βj − λ = 0, ∀x .

is a CONVEX POLYTOPE!

and so, computing θd is solving a Linear Program!

and one has f ∗ ≥ ρd ≥ θd for all d .

Jean B. Lasserre semidefinite characterization

Page 65: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

However as already mentionedFor most easy convex problems (except LP) finiteconvergence is impossible!Other obstructions to exactness occur

Typically, if K is the polytope {x : gj(x) ≥ 0, j = 1, . . . ,m} andf ∗ = f (x∗) with gj(x)∗ = 0, j ∈ J(x∗), then finite convergence isimpossible as soon as the exists x 6= x∗ with J(x) = J(x∗) (xnot necessarily in K)

Jean B. Lasserre semidefinite characterization

Page 66: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

However as already mentionedFor most easy convex problems (except LP) finiteconvergence is impossible!Other obstructions to exactness occur

Typically, if K is the polytope {x : gj(x) ≥ 0, j = 1, . . . ,m} andf ∗ = f (x∗) with gj(x)∗ = 0, j ∈ J(x∗), then finite convergence isimpossible as soon as the exists x 6= x∗ with J(x) = J(x∗) (xnot necessarily in K)

Jean B. Lasserre semidefinite characterization

Page 67: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

A less brutal simplification

With k ≥ 1 FIXED, consider the LESS BRUTALSIMPLIFICATION of

ρd = maxu≥0,λ

{λ : f −∑α,β

uαβm∏

j=1

gjαj (1− gj)

βj − λ ≥ 0, on Rn

|α+ β| ≤ 2d}

to ...

ρkd = max

u≥0,λ{λ : f −

∑α,β

uαβm∏

j=1

gjαj (1− gj)

βj − λ = σ, on Rn

|α+ β| ≤ 2d ; σ SOS of degree at most 2k}

Jean B. Lasserre semidefinite characterization

Page 68: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

A less brutal simplification

With k ≥ 1 FIXED, consider the LESS BRUTALSIMPLIFICATION of

ρd = maxu≥0,λ

{λ : f −∑α,β

uαβm∏

j=1

gjαj (1− gj)

βj − λ ≥ 0, on Rn

|α+ β| ≤ 2d}

to ...

ρkd = max

u≥0,λ{λ : f −

∑α,β

uαβm∏

j=1

gjαj (1− gj)

βj − λ = σ, on Rn

|α+ β| ≤ 2d ; σ SOS of degree at most 2k}

Jean B. Lasserre semidefinite characterization

Page 69: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Why such a simplification?

With k fixed, ρkd = f ∗ as d →∞.

Computing ρkd is now solving an SDP (and not an LP any

more!)However, the size of the LMI constraint of this SDP is

(n+kn

)(fixed) and does not depend on d !For convex problems where f and −gj are SOS-CONVEXpolynomials, the first relaxation in the hierarchy is exact,that is, ρk

1 = f ∗ (never the case for the LP-hierarchy)

• A polynomial f is SOS-CONVEX if its Hessian ∇2f (x) factorsas L(x)L(x)T for some polynomial matrix L(x). For instance,separable polynomials f (x) =

∑ni=1 fi(xi), with convex fi ’s are

SOS-CONVEX.

Jean B. Lasserre semidefinite characterization

Page 70: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Why such a simplification?

With k fixed, ρkd = f ∗ as d →∞.

Computing ρkd is now solving an SDP (and not an LP any

more!)However, the size of the LMI constraint of this SDP is

(n+kn

)(fixed) and does not depend on d !For convex problems where f and −gj are SOS-CONVEXpolynomials, the first relaxation in the hierarchy is exact,that is, ρk

1 = f ∗ (never the case for the LP-hierarchy)

• A polynomial f is SOS-CONVEX if its Hessian ∇2f (x) factorsas L(x)L(x)T for some polynomial matrix L(x). For instance,separable polynomials f (x) =

∑ni=1 fi(xi), with convex fi ’s are

SOS-CONVEX.

Jean B. Lasserre semidefinite characterization

Page 71: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Why such a simplification?

With k fixed, ρkd = f ∗ as d →∞.

Computing ρkd is now solving an SDP (and not an LP any

more!)However, the size of the LMI constraint of this SDP is

(n+kn

)(fixed) and does not depend on d !For convex problems where f and −gj are SOS-CONVEXpolynomials, the first relaxation in the hierarchy is exact,that is, ρk

1 = f ∗ (never the case for the LP-hierarchy)

• A polynomial f is SOS-CONVEX if its Hessian ∇2f (x) factorsas L(x)L(x)T for some polynomial matrix L(x). For instance,separable polynomials f (x) =

∑ni=1 fi(xi), with convex fi ’s are

SOS-CONVEX.

Jean B. Lasserre semidefinite characterization

Page 72: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Why such a simplification?

With k fixed, ρkd = f ∗ as d →∞.

Computing ρkd is now solving an SDP (and not an LP any

more!)However, the size of the LMI constraint of this SDP is

(n+kn

)(fixed) and does not depend on d !For convex problems where f and −gj are SOS-CONVEXpolynomials, the first relaxation in the hierarchy is exact,that is, ρk

1 = f ∗ (never the case for the LP-hierarchy)

• A polynomial f is SOS-CONVEX if its Hessian ∇2f (x) factorsas L(x)L(x)T for some polynomial matrix L(x). For instance,separable polynomials f (x) =

∑ni=1 fi(xi), with convex fi ’s are

SOS-CONVEX.

Jean B. Lasserre semidefinite characterization

Page 73: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Why such a simplification?

With k fixed, ρkd = f ∗ as d →∞.

Computing ρkd is now solving an SDP (and not an LP any

more!)However, the size of the LMI constraint of this SDP is

(n+kn

)(fixed) and does not depend on d !For convex problems where f and −gj are SOS-CONVEXpolynomials, the first relaxation in the hierarchy is exact,that is, ρk

1 = f ∗ (never the case for the LP-hierarchy)

• A polynomial f is SOS-CONVEX if its Hessian ∇2f (x) factorsas L(x)L(x)T for some polynomial matrix L(x). For instance,separable polynomials f (x) =

∑ni=1 fi(xi), with convex fi ’s are

SOS-CONVEX.

Jean B. Lasserre semidefinite characterization

Page 74: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Importantly, if a certain moment-matrix at an optimal solution ofthe the dual HAS RANK 1 then the SDP-relaxation IS EXACT.

Jean B. Lasserre semidefinite characterization

Page 75: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Some preliminary numerical experiments

NON CONVEX quadratic polynomials on a Simplex.

minx{ xT A x : eT x ≤ 1, x ≥ 0 },

where A ∈ Rn×n is randomly generated and then one imposes rnegative eigenvalues and n − r positive eigenvalues in itsspectral decomposition. We have chose r = n/10 andn = 10,20,40,50 and n = 100.

→ In all examples k = 1, that is σ is a degree-2 SOS and inmost examples the rank condition is satisfied and one obtainsthe optimal value with d = 2 and much faster than withGloptiPoly (when it can solve it).→ Solving the second relaxation for instances with n = 50requires 34 s. of CPU, and 1600 s. with n = 100.→ ... No sparsity pattern is exploited.

Jean B. Lasserre semidefinite characterization

Page 76: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Some preliminary numerical experiments

NON CONVEX quadratic polynomials on a Simplex.

minx{ xT A x : eT x ≤ 1, x ≥ 0 },

where A ∈ Rn×n is randomly generated and then one imposes rnegative eigenvalues and n − r positive eigenvalues in itsspectral decomposition. We have chose r = n/10 andn = 10,20,40,50 and n = 100.

→ In all examples k = 1, that is σ is a degree-2 SOS and inmost examples the rank condition is satisfied and one obtainsthe optimal value with d = 2 and much faster than withGloptiPoly (when it can solve it).→ Solving the second relaxation for instances with n = 50requires 34 s. of CPU, and 1600 s. with n = 100.→ ... No sparsity pattern is exploited.

Jean B. Lasserre semidefinite characterization

Page 77: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

Some preliminary numerical experiments

NON CONVEX quadratic polynomials on a Simplex.

minx{ xT A x : eT x ≤ 1, x ≥ 0 },

where A ∈ Rn×n is randomly generated and then one imposes rnegative eigenvalues and n − r positive eigenvalues in itsspectral decomposition. We have chose r = n/10 andn = 10,20,40,50 and n = 100.

→ In all examples k = 1, that is σ is a degree-2 SOS and inmost examples the rank condition is satisfied and one obtainsthe optimal value with d = 2 and much faster than withGloptiPoly (when it can solve it).→ Solving the second relaxation for instances with n = 50requires 34 s. of CPU, and 1600 s. with n = 100.→ ... No sparsity pattern is exploited.

Jean B. Lasserre semidefinite characterization

Page 78: Bounded degree SOS hierarchy for polynomial optimization B. Lasserr… · inConvex Algebraic Geometry(e.g. semidefinite representation of convex sets, algebraic degree of semidefinite

THANK YOU!!

Jean B. Lasserre semidefinite characterization