Conic Formulation for l p -Norm Optimization

journal of optimization theory and applications: Vol. 122, No. 2, pp. 285–307, August 2004 (© 2004)

Conic Formulation for lppp-Norm Optimization1

F. Glineur2 and T. Terlaky3

Communicated by Z. Q. Luo

Abstract. In this paper, we formulate the lp-norm optimization prob-lem as a conic optimization problem, derive its duality properties(weak duality, zero duality gap, and primal attainment) using stan-dard conic duality and show how it can be solved in polynomial timeapplying the framework of interior-point algorithms based on self-concordant barriers.

Key Words. Duality theory, lp-norm optimization, conic optimiza-tion, interior-point methods, self-concordant barrier.

1. Introduction

lp-norm optimization problems form an important class of convexproblems, which includes as special cases linear optimization, quadraticallyconstrained quadratic optimization, and lp-norm approximation problems.

A primal–dual pair of feasible lp-norm optimization problems satisfiesnaturally the weak duality property (as is generally the case between a pri-mal optimization problem and any reasonably formulated dual problem).Moreover, convexity of the problems implies strong duality in the presenceof a Slater point.

However, lp-norm optimization possesses better duality propertiesthan a general convex problem: one can prove indeed that the optimum

1This research has been carried out in part at Delft University of Technology, whilethe first author was visiting the SSOR department, supported by a travel grant from theCommunaute Francaise de Belgique. This author thanks Professor Kees Roos and theSSOR department for their kind hospitality.

2Postdoctoral Researcher, Service de Mathematique et de Recherche Operationnelle,Faculte Polytechnique de Mons, Mons, Belgium. This author was supported by a grantfrom FNRS (Belgian National Fund for Scientific Research).

3Professor, Canadian Research Chair in Optimization, Department of Computing and Soft-ware, McMaster University, Hamilton, Ontario, Canada. This author was supported inpart by Grant OPG0048923 of the National Sciences and Engineering Research Councilof Canada.

2850022-3239/04/0800-0285/0 © 2004 Plenum Publishing Corporation

286 JOTA: VOL. 122, NO. 2, AUGUST 2004

duality gap is always equal to zero and that at least one feasible solu-tion attains the optimum primal objective, even in the absence of a Slaterpoint. These results were presented first by Peterson and Ecker (Refs. 1–3)and later simplified by Terlaky (Ref. 4), using standard convex duality the-ory (i.e., the convex Farkas theorem).

The aim of this paper is to derive these results in a completelydifferent setting, using the machinery of conic convex duality. This newapproach has the advantage of further simplifying the proofs and givingsome insight about why this class of problems has better properties thana general convex problem. We show also that this class of optimizationproblems can be solved up to a given accuracy in polynomial time, usingthe theory of self-concordant barriers in the framework of interior-pointalgorithms (Ref. 5).

1.1. Problem Definition. Let

K={1,2, . . . , r} , I ={1,2, . . . , n} ,

and let {Ik}k∈K be a partition of I into r classes. Problem data are givenby two matrices A∈Rm×n and B ∈Rm×r (whose columns are denoted byai , i ∈ I , and bk, k ∈K) and four column vectors η ∈Rm, c ∈Rn, d ∈Rr ,

p∈Rn such that pi > 1,∀i ∈ I . The primal lp-norm optimization problemconsists in maximizing a linear function of a column vector y ∈Rm undera set of constraints involving lp-norms of linear forms,

(P) sup ηTy,

s.t.∑i∈Ik

(1/pi)|ci −aTi y|pi ≤dk−bT

k y, ∀k∈K.

This formulation is general enough to include many well-studied classes ofconvex optimization problems (Ref. 4) such as linear optimization (n=0),linearly and quadratically constrained quadratic optimization (pi = 2, ci =0), lp-norm approximation (bk = 0); it can even be used to approximategeometric optimization problems (Ref. 6, Chapter 8).

Defining a vector q ∈Rn such that

1/pi +1/qi =1, for all i ∈ I ,

the well-known dual problem for (P) consists in optimizing a highly non-linear objective depending on two column vectors x∈Rn and z∈Rr over a

JOTA: VOL. 122, NO. 2, AUGUST 2004 287

feasible region defined mostly4 by linear equalities and nonnegativity con-straints,

(D) inf ψ(x, z)= cTx+dTz+ ∑k∈Kzk>0

z1−qik

∑i∈Ik

(1/qi)|xi |qi ,

s.t. Ax+Bz=η, z≥0,zk=0⇒xi =0, ∀i ∈ Ik.

1.2. Conic Optimization. We recall here the basics of conic optimiza-tion; a more detailed account can be found in Refs. 7–8. A set is a convexcone if and only if it is closed under addition and nonnegative scalar mul-tiplication. In order to avoid some technical nuisances, all the cones con-sidered in this paper are closed, solid (i.e., with a nonempty interior) andpointed (i.e., containing no lines). Given such a cone C, a m×n matrix A,and two column vectors b and c belonging respectively to Rm and Rn, theproblem of finding a vector x∈Rn that minimizes the linear form cTx overthe intersection of C and the affine subspace

{x ∈Rn|Ax=b} is called the

primal conic problem,

(CP) infx

cTx,

s.t. Ax=b, x ∈C.The vectors that have a nonnegative inner product with every point

of C form another cone, called the dual cone C∗. It is also convex, closed,solid, pointed and allows us to write a dual conic problem involving twovectors of variables y ∈Rm and s ∈Rn,

(CD) sup(y,s)

bTy,

s.t. ATy+ s= c, s ∈C∗.This dual problem involves also optimizing a linear function over theintersection of a convex cone and an affine subspace (the differences withthe primal problem being the direction of the optimization, maximiza-tion instead of minimization, and the description of the affine subspace,a translation of the range space of AT instead of a translation of the nullspace of A).

4The feasible region is slightly modified by a convention taken to handle the case whenone or more components of z are equal to zero: the corresponding terms are left out ofthe sum in the third term of the objective function (to avoid division by zero) and theassociated components of x have to be equal to zero.


Conic optimization is equivalent completely to convex optimization(Ref. 5), i.e., minimization of a convex function over a convex set. How-ever, formulating a convex problem in a conic way has the advantage ofproviding a very symmetric form for the dual problem and reveals oftennew insights about its structure. Duality theorems relate problems (CP)and (CD) to each other.

Theorem 1.1. Weak Duality. Every primal feasible solution x anddual feasible solution (y, s) satisfy the inequality bTy≤ cTx, equalityoccurring if and only if the orthogonality condition xTs=0 holds.

Denoting the optimum objective values5 of problems (CP) and (CD)by p∗ and d∗, this theorem implies that the duality gap p∗−d∗ is nonneg-ative.

A feasible primal (or dual) solution that belongs to the interior ofthe primal (or dual) cone is called a strictly feasible solution. The primal[resp. dual] problem is unbounded when p∗=−∞ [resp. d∗=+∞], infeasi-ble when there is no feasible solution, i.e., when p∗=+∞ [resp. d∗=−∞],and attained when its optimum objective value p∗ [resp. d∗] is achievedby at least one feasible primal [resp. dual] solution (obviously, infeasibleand unbounded problems cannot be attained). The existence of a strictlyfeasible (Slater) solution is critical to guarantee an even stronger relationbetween the primal and dual problems.

Theorem 1.2. Strong Duality. When the dual [resp. primal] problemadmits a strictly feasible solution, equality p∗ = d∗ holds and the primal[resp. dual] problem is either attained or infeasible.

The results described in this section can be extended easily to the caseof several conic constraints involving disjoint sets of variables (using theCartesian product of these cones).

1.3. Structure of the Paper. The rest of this paper is organized asfollows. We define in Section 2 an appropriate convex cone that allowsus to express lp-norm optimization problems as conic programs. We studyalso some aspects of this cone (closedness, interior, dual). We are then inposition to formulate the primal–dual pair (P)–(D) using a conic formu-lation and apply in Section 3 the standard conic duality theory to prove

5Note that, although the feasible set is closed, it may happen that the infimum in (CP) orthe supremum in (CD) are not attained; an example of this situation is given in Section3.3.


the above-mentioned duality results about lp-norm optimization. Section 4studies the algorithmic complexity of interior-point methods applied toour formulation, namely providing a self-concordant barrier for our cones.We conclude with some remarks in Section 5.

2. Cones for lppp-Norm Optimization

The following Lp cone allows us to give a conic formulation forlp-norm optimization.

2.1. Lp Cone.

Definition 2.1. Let n∈N and p ∈Rn with pi > 1. We define the fol-lowing set:

Lp={(x, θ, κ)∈Rn×R+×R+

∣∣∣∣∣n∑i=1

|xi |pi/(piθ

pi−1)≤κ}.

In the case of a zero denominator, the following convention6 is used:

|xi |/0=+∞, when xi �=0,

|xi |/0=0, when xi =0.

Theorem 2.1. Lp is a convex cone.

Proof. Define the function

gp : Rn →R+ :x →n∑i=1

(1/pi

) |xi |pi ,which is convex since y →|y|α is convex for all α≥1. The positively homo-geneous function generated by gp(x)+ δ(θ |1) is

fp : Rn×R+ →R+∪{+∞} : (x, θ) → θgp (x/θ)

=n∑i=1

|xi |pi/(piθ

pi−1)

6This convention means essentially that (x,0, κ)∈Lp⇔x ∈{0}n.


and is also convex (Ref. 9, Section 5, page 35); this definition impliesfp(x,0)= δ(x|0), in accordance with the convention introduced above todeal with a zero denominator. Finally, observing that

epi fp={(x, θ, κ)∈Rn×R+×R |fp(x, θ)≤κ}=Lp,we have that Lp is the epigraph of a convex positively homogeneous func-tion, hence a convex cone.

In order to characterize strictly feasible points, we describe theinterior of this cone.

Theorem 2.2. The cone Lp is solid and its interior is given by

int Lp={(x, θ, κ)∈Rn×R++×R++

∣∣∣∣∣n∑i=1

|xi |pi/(piθ

pi−1)<κ}.

Proof. The interior of an epigraph can be described by Lemma 7.3in Ref. 9 stating that

int Lp= int epi fp={(x, θ, κ) | (x, θ)∈ int dom fp and fp(x, θ)<κ}.

The description of int Lp follows from

int dom fp=Rn×R++.

Solidness of Lp is obtained noting that7

(e,1, n)∈ int Lp.

2.2. Dual Cone.

Theorem 2.3. Dual of Lp. Let p,q ∈ Rn++ such that 1/pi + 1/qi =1 for each i. The dual of cone Lp is the switched cone Lqs defined by(x∗, θ∗, κ∗)∈Lqs ⇔ (x∗, κ∗, θ∗)∈Lq .

Proof. Let us suppose that

v= (x, θ, κ)∈Lp, v∗ = (x∗, θ∗, κ∗)∈Lqs .

7In this paper e stands for the n-dimensional vector with elements all equal to one.


When θ =0,

vTv∗ =κκ∗ ≥0.

Similarly, if κ∗ =0,

vTv∗ = θθ∗ ≥0.

When both θ and κ∗ are positive, we have

vT v∗ ≥xT x∗ + θκ∗n∑i=1

(1/qi)∣∣x∗i /κ∗∣∣qi +κ∗θ

n∑i=1

(1/pi)|xi/θ |pi

≥xT x∗ + θκ∗n∑i=1

[−(xi/θ)(x∗i /κ∗)]=0. (1)

The definitions of Lp and Lqs imply the first inequality. The weightedarithmetic–geometric mean theorem implies

(1/p)|y|p+ (1/q)|y∗|q ≥yy∗, for all y, y∗ ∈R,

with equality if and only if

|y|p=|y∗|q and yy∗ ≥0.

Applying this result with

y=xi/θ and y∗ =−x∗i /κ∗

to each term in (1) gives the second inequality. Thus, we have always

vT v∗ ≥0,

which implies that

Lqs ⊆ (Lp)∗.Suppose now that v∗ = (x∗, θ∗, κ∗) ∈ (Lp)∗, which means that vT v∗

has to be nonnegative for all v = (x, θ, κ) ∈ Lp. In particular, choosing(0,0,1) ∈Lp leads to κ∗ ≥0. On the one hand, when κ∗ =0, choosing v=(x,1, fp(x,1))∈Lp implies that

xTx∗ + θ∗ ≥0, for any x ∈Rn,


which requires x=0 and θ∗≥0 and thus v∗ ∈Lqs . On the other hand, whenκ∗> 0, we can choose x and θ > 0 such that the second inequality in (1)is satisfied with equality, i.e.,

xTx∗ + θfq(x∗, κ∗)+κ∗fp(x, θ)=0.

Choosing now κ = fp(x, θ) so that (x, θ, κ)∈Lp, the inequality vTv∗ ≥ 0can be written as

xTx∗ + θθ∗ +κκ∗ ≥xTx∗ + θfq(x∗, κ∗)+κ∗fp(x, θ),which is seen easily to imply θ∗ ≥ fq(x∗, κ∗), i.e., v∗ ∈Lqs and thus Lqs =(Lp)∗.

The dual of a Lp cone is thus equal, up to a permutation of twovariables, to another Lp cone with a dual vector of exponents. Symmetrybetween Lp and Lqs and between p and q implies also the following equiv-alent identities:

(Lps )∗ =Lq, (Lq)∗ =Lps , (Lqs )∗ =Lp.

Corollary 2.1. Lp and Lqs are solid, pointed, and closed.

Proof. Solidness of Lp, proved earlier, implies solidness of Lqs forsymmetry reasons. Since pointedness is the property dual to solidness, not-ing that Lp = (Lqs )∗ and Lqs = (Lp)∗ is enough to prove that Lp and Lqsare also pointed. To prove closedness8, we start with (Lp)∗ =Lqs and takethe dual of both sides to find ((Lp)∗)∗ = (Lqs )∗. Since (Lqs )∗ = Lp and((Lp)∗)∗ = clLp (Ref. 9, p. 121), we have clLp =Lp, hence Lp is closed.The switched cone Lqs is obviously closed as well.

Note 2.1. Lp cones are not self dual in the general case. However,when pi =2, ∀i, we have

L(2,...,2)={(x, θ, κ)∈Rn×R+×R+

∣∣∣∣∣n∑i=1

x2i ≤2θκ

}

=L(2,...,2)s = (L(2,...,2))∗,usually called the hyperbolic or rotated second-order cone (Refs. 8,10) (itis a linear transformation of the standard second-order cone), which is

8Closedness of Lp can be obtained also from the fact that it is the epigraph of a lowersemicontinuous function (Ref. 9, Theorem 7.1).


self-dual. Another very interesting case of self duality occurs in the three-dimensional case, i.e., when n=1. Namely, writing

(x, θ, κ)∈L(p)⇔|x|p≤pκθp−1

⇔|x|≤p1/pκ1/pθ1/q

⇔|x|q ≤q(pκ)q−1θ/q

⇔(x, θ/q,pκ

)∈L(q)s

shows that L(p) and its dual (L(p))∗ =L(q)s are equal up to a simple scal-ing of the variables (one could define equivalently a scaled inner productand get exact self duality).

To conclude this section, we provide a way to describe the optimalityconditions.

Theorem 2.4. Orthogonality Conditions. The vectors v=(x, θ, κ)∈Lpand v∗= (x∗, θ∗, κ∗)∈Lqs are orthogonal if and only if the following set ofconditions holds:

κ∗(fp(x, θ)−κ)=0, (2a)

θ(fq(x∗, κ∗)− θ∗)=0, (2b)

|xi |pi κ∗qi =|x∗i |qi θpi , (2c)

xix∗i ≤0, for all i. (2d)

Proof. When θ > 0 and κ∗ > 0, a careful examination of the twoinequalities in (1) reveals that each of these conditions is necessary andsufficient (the first two conditions are related to the first equality, whilethe last two conditions come from the second equality). When θ = 0 butκ∗ > 0, we have x = 0 and thus vTv∗ = κκ∗. This quantity is zero if andonly if κ=0, which is equivalent in this case to fp(x, θ)=κ and occurs ifand only if the first condition in (2) is satisfied (all the other conditionsbeing trivially fulfilled). A similar reasoning takes care of the case θ > 0,κ∗ = 0. Finally, when θ = κ∗ = 0, we have x = x∗ = 0 and vTv∗ = 0, whilethe set of conditions (2) is also always satisfied.

3. Duality for the lppp-Norm Optimization Problem

We show how a primal–dual pair of lp-norm optimization problemscan be modeled using the Lp and Lqs cones and derive the relevant dual-ity properties.


3.1. Conic Formulation. In the following, vI [resp. MI ] will denotethe restriction of column vector v [resp. matrix M] to the components[resp. rows] whose indices belong to the set I . Introducing two auxiliaryvectors of variables x∗ ∈Rn and z∗ ∈Rr to represent the linear functionsappearing as arguments of the power functions and in the right-hand sideof the constraints, we can write problem (P) as

sup ηTy,

s.t. ATy+x∗ = c, BTy+ z∗ =d,∑i∈Ik

(1/pi) |x∗i |pi ≤ z∗k, ∀k∈K.

Introducing an additional vector of fictitious variables v∗ ∈Rr whose com-ponents are fixed to 1 by linear constraints, we plug our definition of theLp cone into the formulation and obtain

(Pc) sup ηTy,

s.t. ATy+x∗ = c, BTy+ z∗ =d, v∗ = e,(x∗Ik , v

∗k , z∗k)∈Lpk , for all k∈K,

where we defined for convenience the vectors

pk= (pi | i ∈ Ik), for k∈K;this is exactly a conic optimization problem in the dual9 form (CD), withmultiple conic constraints involving disjoint sets of variables.

Defining a vector q ∈Rn such that

1/pi +1/qi =1, for all i ∈ I,and vectors qk such that

qk= (qi | i ∈ Ik), for k∈K,we derive a dual problem to (Pc) in a completely mechanical way:

(Dc) inf cTx+dTz+ eTv,

s.t. Ax+Bz=η,(xIk , vk, zk)∈Lqks , for all k∈K,

9Therefore, we used an asterisk superscript for our additional variables in order to empha-size their dual nature.


where x ∈Rn, z∈Rr and v ∈Rr are the dual variables. This problem canbe simplified: making the conic constraints explicit, we find

inf cTx+dTz+ eTv,

s.t. Ax+Bz=η,∑i∈Ik|xi |qi

/(qiz

qi−1k

)≤vk, ∀k∈K,z≥0,

keeping in mind the convention on zero denominators that in effectimplies zk = 0⇒ xIk = 0. Finally, we can remove the v variables from theformulation, since they have to be minimized and are constrained onlyby a lower bound, which they will thus attain at any optimal solution.Therefore, we incorporate their optimum values into the objective func-tion, which leads to

inf ψ(x, z)= cTx+dTz+ ∑k∈Kzk>0

z1−qik

∑i∈Ik

(1/qi)|xi |qi

s.t. Ax+Bz=η, z≥0,

zk=0⇒xi =0, ∀i ∈ Ik;this is, not surprisingly, the standard form for a dual lp-norm optimizationproblem.

3.2. Duality Properties. We are now able to prove the weak dualityproperty for the lp-norm optimization problem.

Theorem 3.1. Weak duality. If y is feasible for (P) and (x, z) is fea-sible for (D), we have ψ(x, z)≥ηTy. Equality occurs if and only if, for allk∈K and i ∈ Ik,

zk

[∑i∈Ik

(1/pi)∣∣∣ci −aT

i y

∣∣∣pi +bTk y−dk

]=0, (3a)

xi(ci −aT

i y)≤0, z

qik

∣∣∣ci −aTi y

∣∣∣pi =|xi |qi . (3b)

Proof. Let y and (x, z) be feasible for (P) and (D). Choosing

vk=fqk (xIk , zk), for all k∈K,we have that (x, z, v) is feasible for (Dc), with

cTx+dTz+ eTv=ψ(x, z).


Moreover, choosing (x∗, z∗, v∗) such that the linear constraints in (Pc) aresatisfied, i.e.,

x∗i = ci −aTi y, z∗k =dk−bT

k y, v∗k =1, (4)

we have that (x∗, z∗, v∗, y) is feasible for (Pc). Theorem 1.1 applied to theconic pair (Pc)–(Dc) then states that

cTx+dTz+ eTv≥ηTy,

which in turn implies

ψ(x, z)≥ηTy,

equality occurring if and only if

(x∗Ik , z∗k, v∗k )

T(xIk , zk, vk)=0, for all k∈K.

Since (x∗Ik , v∗k , z∗k)∈Lp

kand (xIk , vk, zk)∈Lq

k

s , Theorem 2.4 states thatthis is equivalent to the following set of conditions for all i∈Ik and k∈K:

zk(fpk (x∗Ik, v∗k )− z∗k)=0, (5a)

v∗k (fqk (xIk , zk)−vk)=0, (5b)

zqik |x∗i |pi =v∗pik |xi |qi , (5c)

xix∗i ≤0. (5d)

The second condition is always satisfied while the other three conditionscan be readily simplified using (4) to give the announced inequalities(3).

Weak duality is a common feature of primal–dual pairs of problems.In contrast to this, the next theorem deals with a strong duality propertythat does not hold in the general convex case.

It is well known that, in the special case of linear optimization, theduality gap at the optimum is always equal to zero and that both optimumobjective values are attained. However, this is not the case when deal-ing with general convex problems, where positive duality gaps and non-attainment can occur. The status of lp-norm optimization lies somewherebetween these two situations: the duality gap is always equal to zero, butattainment of the optimum objective value can be guaranteed only for theprimal problem.

Our proof uses the well-known Goldman–Tucker theorem (Ref. 11)for linear optimization, which we restate here for reference.


Theorem 3.2. See Ref. 11. Assume that the following primal–dualpair of linear optimization problems in standard form is feasible:

min cTx, s.t. Ax=b, x≥0,

max bTy, s.t. ATy+ s= c, s≥0.

Then, there exists a unique optimal partition10 (B,N ) of the index setcommon to vectors x and s such that every optimal solution x to the pri-mal problem satisfies xN =0 and every optimal solution (y, s) to the dualproblem satisfies sB = 0. Moreover, there exists at least one strictly com-plementary optimal primal–dual solution (x, y, s) such that x+ s > 0 andthus xB>0 and sN >0.

The strong duality theorem for lp-norm optimization that we areabout to prove is the following.

Theorem 3.3. Strong Duality. If both problems (P) and (D) are fea-sible, the primal optimal objective value is attained with zero duality gap,i.e.,

p∗ =max ηTy,

s.t.∑i∈Ik

(1/pi)∣∣∣ci −aT

i y

∣∣∣pi ≤dk−bTk y, ∀k∈K,

= inf ψ(x, z),

s.t. Ax+Bz=η, z≥0,

zk=0⇒xi =0, ∀i ∈ Ik.=d∗.

Proof. According to Theorem 1.2, zero duality gap and primalattainment for a primal–dual conic pair are guaranteed by the existenceof a strictly feasible dual solution. Assuming that (x, z) is a feasible solu-tion for (D), we would like to complement it with a vector v such that thecorresponding solution (x, z, v) is strictly feasible for the dual conic formu-lation (Dc). Such a solution would in fact have to satisfy

(xIk , zk, vk)∈ int Lqks , for all k∈K.Using now Theorem 2.2 to identify the interior of the Lqs cones, we

see that both conditions vk>fpk (xIk , zk) and zk>0 have to be valid forall k∈K. Conditions on v do not pose any problem, since its components

10This optimal partition can be computed in polynomial time by interior-point methods.


vk are not restricted by the linear constraints. However, the situation isdifferent for the conditions on z: unfortunately, it is not always possibleto find a strictly positive z, since it may happen that the linear constraintscombined with the nonnegativity constraint on z force one or more ofthe components zk to be equal to zero for all feasible primal solutions.Therefore, we use a three-step strategy, as outlined by the following dia-gram:

(P)≡ (Pc)Weak←→ (Dc) ≡ (D)

(c) ↑↓ ↑↓ (a)

(Pr)Strong (zero gap)

←→ (Dr)

↓ ↑(Attainment) (b) (Strictly feasible)

(a) Since the zero components of z prevent the existence of a strictlyfeasible solution to (Dc), we consider problem (Dr), a restrictedversion of (Dc), where those problematic components of z andthe associated variables x have been removed [hopefully, thisrestricted problem (Dr) will not behave too differently from theoriginal dual (Dc) because the zero components of z and x didnot play a crucial role in it].

(b) Since this restricted dual problem now admits a strictly feasiblesolution, the corresponding primal problem (Pr) has a zeroduality gap and is attained.

(c) The last step consists in converting the optimal zero gap solu-tion of the restricted primal (Pr) into a solution for the originalprimal problem (Pc).

The problematic zk’s are identically equal to zero for all feasible solu-tions. They can be identified by solving the following linear optimizationproblem:

(LP) min 0,s.t. Ax+Bz=η, z≥0.

This problem has the same11 feasible region as our dual problem (D);thus, we are looking for components of z that are equal to zero on

11Actually, its feasible region can be slightly larger from the point of view of the x vari-ables, since the special constraints zk =0⇒xIk =0 have been omitted, but this does nothave any effect on our reasoning.


the whole feasible region of (LP). Moreover, since this problem has aconstant objective function, all its feasible solutions are optimal andwe are actually looking for variables zk that are equal to zero for alloptimal solutions of problem (LP). Writing the dual12 of this problemas

(LD) max ηTy,

s.t. ATy=0, BTy+ z∗ =0, z∗ ≥0,

we observe that both (LP) and (LD) are feasible [the former because (D)is assumed to be feasible, the latter because (y, z∗)= (0,0) is always feasi-ble], which means that the Goldman–Tucker theorem is applicable. There-fore, we can consider the optimal partition (B,N ), which tells us that theset of variables zi that are identically zero on the feasible regions of prob-lems (LP) and (D) is precisely given by index set N . Thus, we are nowready the apply the strategy outlined above.

(a) We introduce the reduced primal–dual pair of lp-norm opti-mization problems where variables zk and xIk with k ∈N havebeen removed. Starting with the reduced dual problem,

(Dr) inf cTIBxIB +dT

BzB+ eTBvB,

s.t. AIBxIB +BBzB=η, (xIk , vk, zk)∈Lqks , ∀ k∈B,

where IB stands for ∪k∈BIk, we can check that it is equivalentcompletely to problem (Dc), since the missing variables zN andxIN , being forced to zero for all feasible solutions, have no con-tribution to the objective or to the linear constraints in (Dc). Thecorresponding conic constraints become (0, vk,0)∈Lqks ⇔ vk ≥ 0,which imply at the optimum that vk = 0, showing that variablesvN can also be removed safely without changing the optimumobjective value. We conclude that inf (Dr)= inf (Dc)= inf (D).

(b) Because of the second part of the Goldman–Tucker theorem,there is at least one feasible solution to (LP) such that zB > 0.Combining the (xIB , zB) part of this solution with a vector vBwith sufficiently large components gives us a strictly feasiblesolution for (Dr) [zk >0 and vk >fqk (xIk , zk) for all k∈B], whichis exactly what we need to apply the strong duality Theorem

12Although problem (LP) is not exactly formulated in the standard form used to stateTheorem 3.2, the same results hold in the case of a general linear optimization problem.


1.2. The primal problem corresponding to (Dr) is

(Pr) sup ηTy,

s.t. ATIBy+x∗IB = cIB , BT

By+ z∗B=dB, v∗B= e,(x∗Ik , v

∗k , z∗k)∈Lpk , ∀k∈B.

Since it is a relaxation of the original problem (Pc) that isassumed to be feasible, Theorem 1.2 implies that it is attainedwith a zero duality gap. Thus, there exists an optimal vector(x∗IB , z

∗B, v∗B, y) such that ηT y=max(Pr)= inf(Dr).

(c) Combining the results obtained so far, we have proved thatmax(Pr)= inf(D). The last step that we need to perform is toprove that max(P) =max(Pr), i.e., that the optimum objectiveof (P) is also attained with the same optimal objective as (Pr).This would be the case if the optimal solution y of (Pr) is alsofeasible for (Pc), but this is not necessarily true hold since (Pr)

is a relaxation (Pc). However, we can overcome this difficulty byperturbing this solution by a suitably chosen vector such that

(i) feasibility for the constraints k∈B is not lost,(ii) feasibility for the constraints k∈N can be gained.

Let us consider (x, z, y, z∗), a strictly complementary solution tothe primal–dual pair (LP)–(LD) whose existence is guaranteed by theGoldman–Tucker theorem. Thus, we have z∗N >0 and z∗B=0. Since all pri-mal solutions have a zero objective, the optimal dual objective value alsosatisfies ηTy=0. Thus, we can write

ηTy=0, ATy=0,

BTBy=−z∗B=0, BT

N y=−z∗N <0.

Considering now y= y+λy with λ≥0 as a solution of (Pc), we computethe value of x∗ and z∗ given by (4), distinguishing the B and N parts (wealready know that v∗ = e):

x∗IB = cIB −ATIBy= cIB −AT

IB y= x∗IB , using ATIB y=0,

z∗B=dB−BTBy=dB−BT

By= z∗B, using BTBy=0,

x∗IN = cIN −ATIN y= cIN −AT

IN y= x∗IN , using ATIN y=0,

z∗N =dN −BTN y=dN −BT

N y+λz∗N , using −BTN y= z∗N .


The conic constraints corresponding to k ∈ B remain valid for allλ, since the associated variables do not vary with λ. Considering nowthe constraints for k ∈ N , we see that x∗IN does not depend on λ,while z∗N can be made arbitrarily large by increasing λ, due to the factthat z∗N > 0. Choosing a sufficiently large λ, we can force (x∗Ik ,1, z

∗k) ∈

Lqks for k ∈ N and thus make (x∗, v∗, z∗, y) feasible for (Pc). Obvi-ously, the vector y is also feasible for (P) with the same objectivevalue.

Evaluating this objective value, we find that

ηTy=ηTy+ληTy=ηTy=max(Pr),

which shows that

sup(P)≥max(Pr).

Combining with our previous results gives

d∗ = inf(D)=ηTy=max(Pr)≤ sup(P)=p∗.Finally, using Theorem 3.1 (i.e., p∗ ≤d∗), we obtain

d∗ = inf(D)=ηTy= sup(P)=p∗,which implies that y is optimum for (P), sup(P)=max(P), and eventuallythe desired result

p∗ =max(P)= inf(D)=d∗.

3.3. Examples. We exemplify the various situations that can arise fora couple of primal–dual lp-norm optimization problems. Letting

r=n=m=1, A=1, B=0,

c=5, d ∈R, η=1, p=3,

one can simplify the primal–dual pair (P)–(D) to

supy1, s.t. |5−y1|≤ 3√

3d1,

inf 5+d1z1+2/(3√z1), s.t. z1>0.

(i) The choice d = 9 leads to a primal optimum equal to y1 = 8.Looking at the dual, one readily sees that (x, z) = (1,1/9) isoptimal with a dual objective value also equal to 8: both theoptimum values are finite and are attained with zero duality gap.


(ii) When d=0, the only feasible solution is y1=5, giving a primaloptimum equal to 5. The dual objective is then inf 5+2/(3

√z1),

which tends to 5 when z1→+∞, but cannot attain it; thus, thereexists problems for which the dual optimum is not attained, i.e.,perfect duality of linear optimization cannot hold for lp-normoptimization (also note there are no strictly feasible primalsolutions in this case, as predicted by Theorem 1.2).

(iii) When d =−1, the primal becomes infeasible, while the dual isunbounded (take z1→+∞).

4. Complexity

The goal of this section is to prove that it is possible to solvean lp-norm optimization problem up to a given accuracy in polynom-mial time. According to the theoretical framework developed by Nes-terov and Nemirovski, finding a computable self-concordant barrier forthe cone C is all that is needed to solve the conic problem (CP) (thedefinition of a self-concordant barrier as well as relevant properties canbe found in Ref. 5). More precisely, we have the following remarkableresult.

Theorem 4.1. Given a self-concordant barrier for the cone C ⊆ Rn

with parameter θ and a feasible interior starting point x0∈ int C, a short-step interior-point algorithm can solve problem (CP) up to ε accuracywithin O

(√θ log

((cT x0−p∗)/ε

))iterations, each iteration requiring the

evaluation of the self-concordant barrier along with its gradient andHessian and the computation of a Newton step by means of a n×n linearsystem.

We describe now a self-concordant barrier that allows us to solveconic problems involving our Lp cone (our approach bears some similar-ities to the one used in Ref. 12). The convex set

{(x, y)∈R×R+| |x|p≤y},

with p>1, admits the well-known self-concordant barrier

rp : R×R++ →R : (x, y) →−2 log y− log(y2/p−x2),

with parameter 4; see Ref. 5, Proposition 5.3.1. Let

n∈N, p∈Rn, pi >1, I ={1,2, . . . , n}.


We have that the set

Sp={(x, y, κ)∈Rn×Rn+×R| |xi |pi ≤yi,∀i∈I, κ=

n∑i=1

yi/pi

}

admits a self-concordant barrier,

sp : Rn×Rn++×R →R :

(x, y, κ) →n∑i=1

(−2 log yi − log(y2/pi

i −x2i )

),

with parameter 4n; Proposition 5.1.2 from Ref. 5 allows us to take theCartesian product of the n smaller sets; adding a variable κ ∈ R leavesthe self-concordant barrier unchanged and taking the intersection withan affine subspace does not influence self-concordancy. We can now useanother result from Nesterov and Nemirovski to find a self-concordantbarrier for the conic hull of Sp, which is defined by

Hp= cl {(x, y, κ, θ)∈Sp×R++|(x/θ, y/θ, κ/θ)∈Sp}= cl

{(x, y, κ, θ)∈Sp×R++| |xi/θ |pi ≤yi/θ, ∀i ∈ I,

κ/θ =n∑i=1

yi/(piθ)

}

={(x, y, κ, θ)∈Rn×Rn+×R×R+|

|xi |pi /θpi−1≤yi,∀i ∈ I, κ=n∑i=1

yi/pi

}

(to check the last equality, one has to consider accumulation points withθ=0, which in fact must satisfy x=0, which in turn can be seen to matchexactly the convention about zero denominators chosen in Definition 2.1),and find that

hp : Rn×Rn++×R×R++ →R : (x, y, k, θ)

→400(sp(x/θ, y/θ)−8n log θ

)is a self-concordant barrier for Hp with parameter 3200n (see Ref. 5,Proposition 5.1.4). The following observation eventually links Hp to ourcone Lp.


Theorem 4.2. The Lp cone is equal to the projection of Hp on thespace of (x, κ, θ); i.e.,

(x, θ, κ)∈Lp⇔∃y ∈Rn+ | (x, y, κ, θ)∈Hp.

Proof. First note that both sets take the same convention in case ofa zero denominator. Let (x, θ, κ)∈Lp. Choosing y such that

yi =|xi |pi /θpi−1 , for all i ∈ I,

ensures that

n∑i=1

yi/pi =n∑i=1

|xi |pi /(piθpi−1)≤κ,

this last inequality because of the definition of Lp. It is now possible toincrease y1 until the equality

κ=n∑i=1

yi/pi

is satisfied, which shows that (x, y, κ, θ) ∈Hp. For the reverse inclusion,suppose that (x, y, κ, θ)∈Hp. This implies that

κ=n∑i=1

yi/pi ≥n∑i=1

|xi |pi /(piθpi−1),

which is exactly the defining inequality of Lp.

In light of the previous theorem, solving problem (CP) with C=Lp,

inf(x,θ,κ)

cTx x+ cT

θ θ + cTκ κ,

s.t. Axx+Aθθ +Aκκ=b, (x, θ, κ)∈Lp,

is equivalent to solving the following conic problem:

inf(x,y,κ,θ)

cTx x+ cT

θ θ + cTκ κ,

s.t. Axx+Aθθ +Aκκ, (x, y, κ, θ)∈Hp,

for which we know a self-concordant barrier with parameter O(n). Thus,one can find an approximate solution to this problem with accuracy ε in


O(√n log 1

ε

)iterations. Moreover, since it is possible to compute the val-

ues of hp and its gradient and Hessian in polynomial time, the total timerequired to solve this problem is polynomial. Generalization to the case ofseveral Lp cones is straightforward and shows eventually that any primalor dual lp-norm optimization problem can be solved up to a given accu-racy in polynomial time.

5. Concluding Remarks

In this paper, we formulated lp-norm optimization problems in aconic way and applied results from standard conic duality theory to derivetheir duality properties. This was an example of nonsymmetric conic dual-ity, i.e., involving cones that are not self dual, unlike the very well-studiedcases of linear, second-order, and semidefinite optimization.

In our opinion, the conic approach leads to clearer proofs, allowingthe specificity of the class of problems under study to be confined to theconvex cone used in the formulation. Moreover, the fundamental reasonwhy this class of optimization problems has somewhat stronger dualityproperties than a general convex problem becomes apparent: this is essen-tially due to the existence of a strictly interior dual solution (bearing inmind that a reduction procedure involving an equivalent regularized prob-lem has to be introduced when the original dual lacks a strictly feasiblepoint).

In the special case where all pi ’s are equal, one might think that it ispossible to derive those duality results with a simpler formulation relyingon the standard cone involving p-norms, i.e., the p-order cone defined as

Lnp={(x, κ)∈Rn×R+| ‖x‖p≤κ}.However, it seems to be impossible to reach that goal, the reason beingthat the homogenizing variables θ and κ∗ appear to play a significant rolein our developments and cannot be avoided.

Another advantage of this approach is the ease with which onecan prove polynomial complexity for our problems: such a proof boilsdown to exhibiting a suitable self-concordant barrier (note that a differentapproach leads to an analogous result in Ref. 13).

A similar treatment can be applied to the class of geometric opti-mization problems (Ref. 14), although with some differences. Duality forgeometric optimization is indeed a little weaker, since primal attainmentcannot be guaranteed. Comparing the definition of Lp with its counter-part used in the case geometric optimization reveals some similarity, butalso a fundamental difference: the geometric cone is not the epigraph of a


convex function, which prevents the application of the procedure describedin Section 4 to derive a self-concordant barrier; see however Ref. 15 for analternative approach that can potentially alleviate this restriction.

A promising research direction consists in trying to generalize ourapproach for lp-norm and geometric optimization to a wider class ofproblems and design primal–dual algorithms that take advantage of theunderlying symmetry of the conic formulation; see Ref. 6, Chapter 7 fora first step in that direction.

References

1. Peterson, E. L., and Ecker, J. G., Geometric Programming: Duality in Qua-dratic Programming and lp-Approximation, I, Proceedings of the InternationalSymposium on Mathematical Programming, Edited by H. W. Kuhn and A. W.Tucker, Princeton University Press, Princeton, New Jersey, pp. 445–480, 1970.

2. Peterson, E. L., and Ecker, J. G., Geometric Programming: Duality inQuadratic Programming and lp-Approximation, II, SIAM Journal on AppliedMathematics, Vol. 13, pp. 317–340, 1967.

3. Peterson, E. L., and Ecker, J. G., Geometric Programming: Duality inQuadratic Programming and lp-Approximation, III, Journal of MathematicalAnalysis and Applications, Vol. 29, pp. 365–383, 1970.

4. Terlaky, T., On lp-Programming, European Journal of Operational Research,Vol. 22, pp. 70–100, 1985.

5. Nesterov, Y. E., and Nemirovski, A. S., Interior-Point Polynomial Methodsin Convex Programming, SIAM Studies in Applied Mathematics, SIAMPublications, Philadelphia, Pennsylvania, 1994.

6. Glineur, F., Topics in Convex Optimization: Interior-Point Methods, ConicDuality, and Approximations, PhD Thesis, Faculte Polytechnique de Mons,Mons, Belgium, 2001.

7. Stoer, J., and Witzgall, C., Convexity and Optimization in Finite Dimensions,I, Springer Verlag, Berlin, Germany, 1970.

8. Sturm, J. F., Primal–Dual Interior-Point Approach to Semidefinite Program-ming, PhD Thesis, Erasmus University Rotterdam, Rotterdam, Netherlands,1997.

9. Rockafellar, R. T., Convex Analysis, Princeton University Press, Princeton,New Jersey, 1970.

10. Boyd, S., Lobo, M. S., Vandenberghe, L., and Lebret, H., Applications ofSecond-Order Cone Programming, Linear Algebra and Applications, Vol. 284,pp. 193–228, 1998.

11. Goldman, A. J., and Tucker, A. W., Theory of Linear Programming, LinearEqualities and Related Systems, Edited by H. W. Kuhn and A. W. Tucker,Annals of Mathematical Studies, Princeton University Press, Princeton, NewJersey, Vol. 38, pp. 53–97, 1956.


12. Xue, G., and Ye, Y., An Efficient Algorithm for Minimizing a Sum ofp-Norms, SIAM Journal on Optimization, Vol. 10, pp. 551–579, 2000.

13. Roos, C., den Hertog, D., Jarre, F., and Terlaky, T., A Sufficient Conditionfor Self-Concordance, with Application to Some Classes of Structured ConvexProgramming Problems, Mathematical Programming, Vol. 69, pp. 75–88, 1995.

14. Glineur, F., Proving Strong Duality for Geometric Optimization Using a ConicFormulation, Annals of Operations Research, Vol. 105, pp. 155–184, 2001.

15. Glineur, F., An Extended Conic Formulation for Geometric Optimization,Foundations of Computing and Decision Sciences, Vol. 25, pp. 161–174, 2000.

Conic Formulation for l p -Norm Optimization

Documents