Journal of Global Optimization 7: 143-182, 1995. 1 4 3 9 1995 KluwerAcademicPublishers. Printed in theNetherlands. Finding All Solutions of Nonlinearly Constrained Systems of Equations COSTAS D. MARANAS and CHRISTODOULOS A. FLOUDAS* Department of Chemical Engineering, Princeton University, Princeton, NJ 08544-5263, U.S.A. (Received: 15 December 1994; accepted: 15 May 1995) Abstract. A new approach is proposed for finding all e-feasible solutions for certain classes of nonlinearly constrained systems of equations. By introducing slack variables, the initial problem is transformed into a global optimization problem (P) whose multiple global minimum solutions with a zero ob jective value (ff any) correspond to all solutions of the initial constrained system of equalities. A l l e-globally optimal points of (P) are then localized within a set of arbitrarily small disjoint rectangles. This is based on a branch and bound type global optimization algorithm which attains finite e-convergence to each of the multiple global minima of (P) through the successive refinement of a convex relaxation of the feasible region and the subsequent solution of a series o f nonlinear convex optimization problems. Based on the form of the participating functions, a number of techniques for constructing this convex relaxation are proposed. By taking advantage of the properties of products of univariate functions, customized convex lower bounding functions are introduced for a large number of expressions that are or can be transformed into products of univariate functions. Alternative convex relaxation procedures involve either the difference of two convex functions employed in aB B [23] or the exponential variable transformation based underestimators employed for generalized geometric programming problems [24]. The proposed approach is illustrated with several test problems. For some of these problems additional solutions are identified that existing methods failed to locate. Key words: Global optimization, nonlinear systems of equations, all solutions. 1. Introduction A fundamental task in applied mathematics, engineering and sciences is finding all solutions of a set of equations. This task is sometimes further complicated by requiring the simultaneous satisfaction of a number of inequality and/or variable bound constraints. Not only the problem of computing all solutions of nonlinearly constrained systems of equations is NP-hard, but it is also possible that there exists exponentially many such solutions [1]. In addition, simply checking if a solution exists is NP-hard [2]. There exists a large body of literature on methods for solving systems of equations. These methods fall within the following three broad classes: (i) Newton and quasi-Newton type methods; (ii) homotopy continuation type methods; and (iii) interval-Newton methods. New ton and quasi-Newton type methods and their modifications achieve super- li near conver gence only w hen they are well within t he neighborhood of the solution. However, these methods are likely to fail if the initial guess is poor, or if singu-
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Journal o f Global O ptimization 7: 143-182, 1995. 1439 1995 Kluw erAc ade micP ublishe rs. Printed in theNetherlands.
Finding All Solutions o f N onlinearly C onstrained
Sys tems of Equ ations
C O S T A S D . M A R A N A S and C H R I S T O D O U L O S A . F L O U D A S *Department o f Chem ical Engineering, Princeton University, Princeton, N J 08544-5263, U.S.A.
(Received: 15 D ece m ber 1994; accepted: 15 M ay 1995)
Abstract . A new approach is proposed for f inding al l e-feasible solut ions for cer tain classes of
nonlinearly constrained system s of equat ions. B y introducing s lack variables , the init ia l pro blem is
t rans formed in to a g loba l op t imiza t ion prob lem (P) w hose mul t ip le g loba l min im um so lu tions wi th a
zero ob ject ive value (ff any) correspond to al l solutions of the ini tial constrained system of equali ties.
A ll e-global ly opt imal points o f (P) are then local ized w ithin a set of arbit rar ily small dis joint
rectangles . This is based on a branch and b ound ty pe global opt imizat ion algo ri thm wh ich atta ins
f ini te e-co nverge nce to each o f the mult iple global m inim a of (P) throug h the successive ref inement of
a conve x relaxat ion of the feasible region and the subsequent solut ion of a ser ies o f nonl inear co nve x
optim izat ion problems. Based o n the form of the partic ipating funct ions, a num ber of techniques forconstruct ing this co nve x relaxat ion are proposed. B y taking advan tage of the properties of products of
un ivariate func t ions , cus tomized con vex low er bounding func t ions a re in t roduced for a l a rge num berof expressions that are or can be t ransformed into products o f univariate funct ions. Alternat ive con vex
relaxa tion procedures invo lve e ither the d i f ference of two convex func t ions employ ed in aB B [23] o r
the exponential var iable t ransforma tion based underestimators em ploy ed for general ized geom etr ic
prog ram m ing problems [24]. The propo sed app roach is i llustrated with several tes t problems. Fo r
som e o f these prob lem s additional solutions are identified that existing meth ods failed to locate.
K e y w o r d s : Glo bal optimization, non linear system s of equations, all solutions.
1 . I n t r o d u c t i o n
A f u n d a m e n t a l t a sk i n a p p l i e d m a t h e m a t ic s , e n g i n e e r in g a n d s c i e n c e s i s f i n d in g
a l l s o lu t io ns o f a s e t o f equa t io ns . Th i s t a s k i s s o m et im es f ur ther co mp l i ca t ed by
requ ir ing t he s im ul t a neo u s s a t is f a c t io n o f a nu m ber o f inequa l i t y a nd /o r v a r ia b le
bo u nd co nst ra in ts . N o t o n ly t he pro b lem o f co mp ut ing a l l so lu t io ns o f no n l inea r ly
co ns t ra ined s y s t ems o f equa t io ns i s NP - ha rd , but i t i s a l s o po s s ib l e t ha t t here
e x i s t s e x p o n e n t i a l l y m a n y s u c h s o l u t i o n s [ 1 ] . I n a d d i t i o n , s i m p l y c h e c k i n g i f a
s o lu t io n ex i s t s i s N P - ha rd [ 2 ]. There ex i s t s a la rge bo d y o f l it era ture o n m et ho ds f o r
s o l v i n g s y s t e m s o f e q u a t i o n s. T h e s e m e t h o d s f a ll w i t h i n th e f o l l o w i n g th r ee b r o ad
c l as s es : ( i) N e w t o n a n d q u a s i - N e w t o n t y p e m e t h o d s ; ( ii ) h o m o t o p y c o n t i n u a t io n
t y pe met ho ds ; a nd ( i i i ) in t erv a l - Newt o n met ho ds .
N e w t o n a n d q u a s i - N e w t o n t y p e m e t h o d s a n d t h ei r m o d i f ic a t i o n s a c h i e v e s u p er -
l in e a r c o n v e r g e n c e o n l y w h e n t h e y a re w e l l w i t h i n th e n e i g h b o r h o o d o f t h e s o lu t io n .
H o w e v e r , t h e s e m e t h o d s a r e l i k e l y t o f a i l i f t h e i n i ti a l g u e s s i s p o o r , o r i f s in g u -
l a r po in ts a re encou n te red . Modi f i ca t ions in an a t t empt to avo id s ingu la r i ti e s m ay
incorpora t e trus t - reg ion t echn iques such as Pow el l ' s "do g leg " metho d [31 ] , s t eep -
es t de sce nt d i rec t ion inform at ion [7], [10] , [26] and a l tera tions on the quasi -N ew tonJacob ian es tima tes [30 ]. Th i s typ e o f me thod s , a l though ve ry co m puta t iona l ly ef fi-
c i en t , cann o t p rov ide guaran tees fo r conv ergenc e . Th i s i s m an i fes t ed in p rac t i ce
wi th the i r poor con verg enc e charac te ri s ti c s .
O n e o f t h e m o s t w i d e l y u s e d m e t h o d fo r l o c a ti n g s o l u ti o n s o f n o n l i n e a r s y s te m s
of equa t ions be longs to the b road c l ass of em bedding methods. Th i s c l a s s o f m e t h o d s
a re a ls o k n o w n a s continuation, homotopy continuation, or incremental loading,
and a re based on the p ioneer ing work o f [19 , 20 , 8 , 18] . Th e bas i c idea o f hom otop y
con t inua t ion methods i s to c rea t e a fam i ly o f a s ing le pa rame te r func t ions so tha t
t h e s o lu t io n fo r ( t = 0 ) i s k n o w n a n d th e n s o l v e a s e q u e n c e o f p ro b l e m s w i t h ts t ead i ly inc reas ing f rom ( t = 0 ) t o ( t = 1 ) u s ing the so lu t ion o f one p rob lem
as an es t ima te fo r t he nex t . A po pu la r va r i a t ion i s to use a sys t em var i ab le a s the
con t inua t ion pa ram ete r and in t eg ra t e the re su lt ing sys t em o f o rd ina ry d i f fe ren t ia l
equa t ions towards s t eady-s t a te by u t i li z ing AU TO [9] . A p rob lem co m m on to
a l l hom otopy va r i an t s i s t ha t va r i ab le bounds and inequ a l i ty cons t ra in t s cann o t be
hand led d i rec t ly . A com prehe ns ive rev iew o f the ex tens ive l i te ra tu re in th is a rea c an
be foun d in [12] . Wh i l e in p rac t i ce hom otopy con t inua t ion m ethod s a re f requen t ly
used in an a t t em pt to loca te a l l so lu t ions o f a rb it ra ry n on l inea r sys t ems o f equa t ions ,
m a them at i ca l guaran tees tha t a l l so lu tions wi l l be foun d ex i s t on ly in spec ia l cases
(e .g . po lyn om ia l sys tems wi th no cons t ra in ts ) . Fo r po lyno m ia l sys tems o f equa t ions ,
how ever , M organ [27 ] p ropose d a d i f fe ren t ia l a rc l eng th con t inua t ion us ing a spec ia l
h o m o t o p y t h a t e s ta b l is h e s a n u m b e r o f c o n t in u a t i o n p a t hs g u a ra n t e e d to c o n v e rg e
t o a l l p o s s ib l e r e a l a n d c o m p l e x ro o t s. Tw o p o p u l a r s o f t w a re p a c k a g e s , C O N S O L
[2 7] a n d P O LS Y S [34 ] h a v e i m p l e m e n t e d t hi s m e t h o d .
In t e rva l -N ew ton m ethods c an f ind rec t ang les con ta in ing a ll so lu tions o f non-
l inea r sys t ems o f equa t ions w i th in ce r t a in va r iab le bounds w i th ma them at i ca l ce r -
ta in ty . T hey do so by app ly ing the c l a ss ica l New ton- l ike it e ra tive method s onin te rva l va r i ab les ra the r than va r i ab les co up led w i th a g enera l i zed b i sec t ion s t ra t e-
gy [29 , 13 ]. A ve rs ion o f t he bas ic In t e rva l -Ne wton m ethod has been im plem en ted
i n to t h e p u b li c d o m a i n s o f t w a re p ro g ra m IN TB IS [1 7] w h i c h i s c o u p l e d w i t h a
por t ab le in t e rva l s t andard func t ion l i b ra ry IN TL IB [16]. T he m ain a t t rac t ive fea -
t u r e o f In t e rv a l -N e w t o n m e t h o d s i s t h a t t h e y p ro v i d e m a t h e m a t i c a l g u a ra n t e e s
fo r co nve rgen ce to a l l so lu t ions o f fa i r ly a rb it ra ry no n l inea r sys t em s o f equ a t ions
wi th in ce r t a in va r iab le bound s . Ho we ver , t h is w ide app l i cab i l it y to a lm os t a rb i tra ry
non l inea r func t ions com es a t an expen se . Bec ause no spec i fi c st ruc tu re o f i nd iv id -
ua l express ions i s ana lyzed the ob ta ined in te rva l boun ds can som et imes be fa i r lyloose .
Th e p ro p o s e d a p p ro a c h is b a s e d o n c o n v e x l o w e r b o u n d i n g c o u p l e d w i t h a p a r-
t it ion ing s t ra t egy and l ike In t e rva l -N ew ton me thods , i t can p rov ide gu aran tees fo r
c o n v e rg e n c e t o a ll e - so lu t io n s . Th e fu n d a m e n t a l d if f e r e n c e, h o w e v e r , b e t w e e n o u r
p roc edure and In t e rva l -New ton m ethods i s t ha t wh i l e the fo rm er u ti li zes a s ing le
v a l u e t o l o w er b o u n d f u n c t i o n s w i t h i n r ec t an g u l a r d o m a i n s , w e l o w er b o u n d n o n -
con vex func t ions wi th con vex func t ions . By exp lo i t ing the ma them at ica l st ruc tu re
of the p rob lem , th is typ ica l ly r esu l t s in mu ch t igh te r bounds . In the ne x t sec t ion , adescr ip t ion o f the p rob lem i s p resen ted .
2 . P r o b l e m D e s c r i p t io n
This pape r addresses the p rob lem of iden t i fy ing a l l so lu t ions o f a non l inear sys tem
of equa t ions sub jec t to inequa l i ty cons t r a in t s and var i ab le boun ds a nd i s fo rmula ted
as:
h j ( x ) = O , jEA/ 'E (S )
gk (x) < o , k ~ X z
XL < X < xU~
w he re AlE is the set o f equal i t ies , .Mz the se t of inequ al i ty const ra ints , and x the
vec to r o f var i ab les. N ote tha t in fo rmula t ion (S ) the to ta l nu m ber o f var i ab les i s
a l low ed to be d i f f e ren t than the to ta l num ber o f equa l i ti es so as ne i ther the ex i s t ence
nor the un iqueness o f a so lu t ion o f (S ) is pos tu la ted . Therefore , bo th over spec i f i ed
and u nder spec i f i ed sys tems a re inc luded in the p resen t inves t iga tions . No te tha t anu m ber o f imp or tan t p rob lem s na tu ra l ly a r ise as spec ia l ins tances o f fo rm ula t ion
(S) . O n o ne hand , by o mi t t ing a l l inequa l i ty cons tr a in ts , (S ) cor responds to a sys tem
of non l inear equa t ions . On the o ther hand , by e l imina t ing a l l equa l i ty cons t ra in t s
(S) che cks the ex i s t ence o f f eas ib le po in t s fo r the g iven inequa l i ty cons t r a in t se t
( feasibi l ity problem ) .
F o r m u l a t io n ( S ) can b e t ran s f o rm ed i n to t h e f o l l o w i n g m i n - m ax o p t im i za t io n
prob lem [15]
m i n m a x I h j ( x ) lx j 6 A ; E
s u b j e c t t o g k ( x ) _< O , k 6 A / )
x L ~ x < x U.
By in t roduc ing a s ing le s l ack var iab le s , t he min-m ax prob lem can be w r i tt en as
146 C . D . M A R A N A S A N D C . A . F L O U D A S
C l ear ly , t h e re i s a o n e t o o n e co r r e sp o n d en ce b e t w een m u l ti p le g l o b a l m i n i m a
(x* , s* ) o f (P0) fo r wh ich s* = 0 and so lu tions o f (S ). Th i s m eans tha t i f t he
g loba l min imum of (P0) invo lves a nonzero s l ack var i ab le s* then the o r ig ina lp rob lem (S) has no so lu t ions . Note tha t , un less the func t ions h ( x ) a n d g k ( x )
are l i near and convex r espec t ive ly , fo rmula t ion (P ) cor responds to a n o n c o n v e x
opt imiza t ion p rob lem. Th i s im pl ies tha t i f a loca l op t imiza t ion approach i s us ed to
so l v e (P 0 ) , o n e m i g h t m i s s so m e o f t h e m u l t ip l e g l o b a l m i n i m a o f ( P 0 ) o r ev en
er roneo us ly ded uce tha t there a re no so lu t ions fo r (S) . Therefore , an ap proach tha t
i s guaran teed to a lways loca te a ll m u l t i p le g l o b a l m i n i m a o f ( P 0 ) ap p ea r s t o b e
nec ess ary for solving (S) so that ( i) the cor rect solut ion vec tor (x* , s*) i s ident i f ied
and ( ii ) a l l solut ions (x*) o f (S ) w i th s* = 0 are fou nd in a l l instances. In th is w ork ,
a de te rmin i s t i c g loba l op t imiza t ion i s p roposed w hich i s guaran teed to loca te a l le - gl o b al m i n i m a o f ( P 0 ) t h r o u g h th e su cces s iv e r e f in em en t o f co n v e r g i n g l o w er
an d u p p e r b o u n d s o n t h e so l u ti o n b a sed o n t h e so l u ti o n o f co n v ex o p t im i za t io n
p r o b l em s d e f i n ed b y a b r an ch an d b o u n d ap p r o ach . A l o w er b o u n d o n t h e so l u t io n
of (P0) i s found by fir st r ep lac ing each non con vex cons t r a in t in (P0) w i th a conv ex
unde res t imat ion o f i t and then f ind ing the so lu t ion o f the con vex r e laxa t ion OR) o f
(P0) w i th com m erc ia l ly ava i l ab le so lver such as MIN OS 5.4 [28] as sh ow n in [22]
and [23] . This approach natural ly par t i t ions the const ra ints of formulat ion (P0)
in to conv ex ( fo r whic h no r e laxa t ion i s r equ i red) and n onc onv ex cons t ra in t s . Th i s
par t it ion ing y ie lds the fo l lowing a l t e rna t ive fo rmula t ion (P ) :
subjec t to
m i n sx,s>O
n o n e _ \hj ( x ) - s <_ O, j E .A/noncE
n o n c- h i (x) - s < O, j E .A/noncE
g~~ _< 0 , kEA/ ' noncI
h~m (x) = O , j e J~nnE
g~ ~ < O , j E A / 'c o n v l
x L _ _ _ x _ < x v
(P )
H e r e J ~ f n o n c E , J ~ f l i n E a r e t he se t s o f nonconvex and l inear equa l i ty cons t r a in t s
resp ectiv ely, a nd Afnonc~,A/'convZ are the sets o f no nc on ve x and co nv ex ineq ual i ty
const raints ,
= A; .onc u mE, X 1 = Y.o .c u X onv .
A con vex r e laxa t ion OR) o f (P ) o f the fo rm,
m i n s ( R )X,s>_O
sub jec t t o ~ , n o nc { _ \ _, ~ + , j ~ x ) - - 8 < O , j E . A / n o n c E
a s t h e s i ze o f t h e r e c t a n g u l a r d o m a i n a p p r o a c h e s z e r o ( 6 = 0 ) . T h i s is i m p o r t a n t f o r
p r o v i n g f in i te e - c o n v e r g e n c e . T h e o r d e r O ( e ) = 0 ( 6 n ) w i t h w h i c h e a p p r o a c h e s
z e r o a s 6 g o e s t o z e ro i s i m p o r t a n t b e c a u s e i t d e t e r m i n e s t h e s p e e d o f c o n v e r g e n c e .C l e a rl y , th e l a rg e s t p o s s i b l e v a l u e f o r n i s d e s i r a b le s o a s th e m a x i m u m t o l e r a n c e
r e a c h e s a n a r b it ra r y v a l u e e f o r a n o t t o o s m a l l v a r i a b le r a n g e 6 . F o r e x a m p l e , i f
t h e m a x i m u m s e p a r a t io n e g o e s a s 6 2 t h e n a v a l u e o f j u s t 6 = 0 .0 1 s u ff ic e s t o m e e t
a c o n v e r g e n c e t o l e r a n c e o f e = 0 . 0 0 0 1 .
A n e f fi c ie n t c o n v e x l o w e r b o u n d i n g o f n o n c o n v e x f u n c t io n a l s a p p e a r in g i n
f o r m u l a t i o n ( P ) i s c le a r ly c e n t r a l t o th e d e s i g n o f th e p r o p o s e d g l o b a l o p t i m i z a -
t i o n a p p r o a c h f o r l o c a t i n g a l l s o lu t i o n s . U n d o u b t e d l y , t h e ti g h t e r th e c o n v e x l o w e r
b o u n d i n g i s t h e b e t t e r t h e q u a l it y o f th e o b t a i n e d l o w e r b o u n d s w i l l b e , a n d c o n s e -
q u e n t l y t h e f as t e r t h e a l g o r i t h m w i l l c o n v e r g e . T h e t i g h te s t p o s s i b l e c o n v e x l o w e r
b o u n d i n g f u n c t io n fo r a n y a r b i tr ar y n o n c o n v e x f u n c t i o n f ( x ) i n s id e s o m e re c t an -
g u l a r r e g i o n P i s c a l l e d t h e convex envelope ~ b(x ) o f f ( x ) , a n d i t m u s t c o n f o r m t o
t h e f o l l o w i n g p r o p e r t i e s [ 15 ]:
( i) ~ b ( x ) c o n v e x f o r a l l x E P .
( i i) f ( x ) _> ~ b ( x ) f o r a l l x E P .
( ii i) F o r a ll f u n c t i o n s 9 ( x ) t h a t s a t i s f y (i ) a n d ( i i ), ~ b (x ) _> g ( x ) f o r a l l x E P .
U n f o r t u n a t e ly , i n a ll b u t t h e s i m p l e s t c a s e s th e r e e x i s ts n o m e t h o d f o r d e r i v in gt h e c o n v e x e n v e l o p e f o r a rb i tr a ry f u n c t i o n s d e f i n e d i n s id e a r b it ra r y d o m a i n s . A s a
r e su l t, t h e f o c u s i n t h i s w o r k i s t o id e n t i fy t h e m a x i m u m p o s s i b l e f u n c t i o n w h i c h
s a t is f ie s p r o p e r t i e s ( i) a n d ( i i) . T h e r e e x i s t s a n u m b e r o f t e c h n i q u e s f o r o b t a i n i n g
f u n c t i o n s t h a t s a t i s f y p r o p e r t i e s ( i ) , ( i i ) . I n t h e f o l l o w i n g s e c t i o n s , a n u m b e r o f
c o n v e x l o w e r b o u n d i n g p ro c e d u r e s a r e d i s c u s s e d w h i c h c a n b e o f u s e n o t o n l y f o r
t h e p r o b l e m o f l o c a t i n g a ll m u l t i p l e s o l u t i o n s b u t a l s o f o r a n y d e t e r m i n i s t i c b r a n c h
a n d b o u n d g l o b a l o p ti m i z a t i o n a lg o r i th m b a s e d o n c o n v e x l o w e r b o u n d i n g . T h e
f ir st c o n v e x l o w e r b o u n d i n g t e c h n i q u e i s m o t i v a t e d b y t h e f a c t th a t a la r g e n u m b e r
o f n o n c o n v e x t e r m s a p p e a r in g i n d i f fe r e n t m o d e l s a r e o r c a n b e t r a n s fo r m e d i n tot h e p r o d u c t o f f u n c t i o n s o f a s i n g l e v a ri a b l e ( u n i v a r ia t e f u n c t i o n s ) . B y e x p l o i t i n g
t h e p r o p e r ti e s o f p r o d u c t s o f u n i v a r i a te f u n c t i o n s , t i g h t c o n v e x l o w e r b o u n d i n g
f u n c t i o n s a r e d e r i v e d i n t h e n e x t s e c t io n .
3 . P r o d u c t s o f U n i v a r i a t e F u n c t i o n s
A f u n c t i o n f : T ~ ~ R o f a s i n g l e v a r i a b l e z i s c a l l e d u n i v a r i a t e f u n c t i o n . P r o d u c t s
o f u n i v a r i a t e f u n c t i o n s f i ,
N
: ( x ) = I - [i=1
a r e i n g e n e r a l n o n c o n v e x f u n c t i o n s e v e n i f t h e c o r r e s p o n d i n g u n i v a r ia t e f u n c -
t i o n s a r e c o n v e x . B y u t i l i z i n g a p p r o p r i a t e l i n e a r t r a n s f o r m a t i o n s , i f n e c e s s a r y , a
l a rg e n u m b e r o f n o n li n e a r it ie s a p p e a r i n g i n a p p l i e d m a t h e m a t i c s a n d e n g i n e e r i n g
p r o b l e m s c a n b e d e s c r i b e d a s p r o d u c t s o f u n i v a r ia t e fu n c t i o n s .
A 1 - K h a y y a l a n d F a l k [3 ] s h o w e d t h a t th e n o n c o n v e x b i li n e a r p r o d u c t o f x yi n s id e t h e r e c t a n g u l a r d o m a i n ] x L , x U I x [yL , yU [ c a n b e t ig h t ly c o n v e x l o w e r
r ~ P q
I . . I I . . I
b o u n d e d b y t h e f o l l o w i n g l i n e a r c u t:
m a x ( x L y q - x y L - - x L y L, x U y + x y U -- x U y U )
F i rs t, t h e c o n d i t i o n s u n d e r w h i c h a s i m i la r r e s u lt h o l d s f o r t h e p r o d u c t o f t w o
a r b i t ra r y u n i v a r i a t e f u n c t i o n s f ( x ) a n d g ( y ) a r e i n v e s t i g a t e d .
T H E O R E M 1. I f f , g a r e t w i ce d if fe r en t ia b l e u n i va r i a te f u n c t i o n s f ( x ) , g ( y ) 6 C2d e f i n e d i n s i d e a r e c t a n g l e [ ( x L , x U ) , ( y L y U ) ] a n d
l ( x , y ) = m a x { r + r f ( x ) ) _ f L g L ,
r + r I ( x ) ) _ f U g U }
w h e r e
f L = i n f f ( x ) ,x L < m < X U
f u = su p f ( x ) ,x L < x < Z U
g L = i n f g ( x ) ,x L < x < x U
g U = sup g ( x )x L < x < x U
a n d r 1 6 2 1 6 2 r a re th e c on ve x
e n v e l o p e s o f t h e u n iv a r ia t e f u n c t i o n s f Lg ( y ) , g Z f ( x ) ' f i g ( y ) , a n d g V f ( x ) r e s p ec -t i ve ly then:
( i) l ( x , y ) i s c o n v e x , V ( x , y ) 6 [ x L , x U ] • [ yL , y U ] .
( g ~ f ( x ) ) - S ~ g ~ a r e c o n v e x as t he s u m o f t he c o n v e x e n v e lo p e s o f u n i v ~ i a t e
f u n c ti o n s. S i n c e t h e m a x i m u m o f t w o c o n v e x f u n c t io n s is a c o n v e x f u n c t io n a s w e l l ,
s t a t e m e n t ( i ) i s t ru e a n d l ( x , y ) i s c o n v e x f o r a ll ( x , y ) i n [ x L , xU ] • [ yL , yU ]
Proof . T h e o r e m ( 1) p r o v e s t h a t fu n c t i o n l ( x , y ) c o n f o r m s w i t h P r o p e r t i e s ( i )a n d ( ii ) o f S e c t i o n 3 . T h e r e f o r e , i t r e m a i n s t o s h o w t h a t i t s a t is f ie s P r o p e r t y ( i ii ) o f
S e c t i o n 3 . B e c a u s e t h e c o n v e x e n v e l o p e o f a u n iv a r ia t e c o n c a v e f u n c t i o n d e f i n e d
i n a n i n te r v a l is th e l i n e s e g m e n t c o n n e c t i n g t h e th e t w o e n d p o i n t s w e h a v e :
r ) g U I f (x ; )-- ( x L ) x U I ( x L ) - x LI ( xU ) }= ~L x + - Z - 7 J '
i y~ 7 + y~ y~ ] '
+ ( f g ( ,,)) : s < ,g ( y u ) _ g ( y L ) x +
y V _ y L
y ~ g ( y L ) _ ~ ,~g ( d . . l ) ]
Y ' - 7 J "
A f t e r s u b s t i tu t i n g t h e s e e x p r e s s i o n s i n t o t h e r e l a t io n f o r l ( x , y ) w e o b t a i n:
l ( x , y ) = m a x (/1 ( x , y ) , / 2 ( x , y ) )
w h e r e
l l ( X , y ) - ~
+
g L I ( X U ) _ f ( x L ) " [ I L U ( y U ) _ g ( y L ) ,- ~ x L X + L - ~ - ~ Y
g L X U f ( x L ) _ x L f ( x U ) y U g ( y L ) _ y L g ( y U )
" - ~ = " ~ ..[_ L y V _ y L _ f L g L ]
r , < . , ( , ' - - ' ) - , ( , ' - . ) 17 7 J ~ + [" 7e yL j Y
+ [ ffy%(yr')-yLn(yU)L + u U _ y L -- U o
B e c a u s e f ( x ) , g ( y ) a r e m o n o t o n i c o n e o f th e f o l l o w i n g a l t e rn a t iv e s i s tr u e:
( a ) I ( x L ) = I L , i ( x U ) = I v ' g ( x L ) = g L , g ( x U ) = g U ,
@ ) f(x L ) = . f L , i ( z u ) = i ~ , g(x L ) = g U , g(x ~ ) = g L ,
( c ) I ( x L ) = I U , i ( z U ) = i s , , g ( z L ) = g L , g ( x t ' ) = g U ,
(d ) f ( x L ) = i v , f ( x U ) = I L , g ( x L ) = g V , g ( x U ) = gL .
1 5 2 C . D . M A R A N A S A N D C . A . FL O U D A S
F i g . 1 .
( x ~ , # ) R ( x% y~
%
(x L # ) (x~,y )
Decom position of rectangleR into two triangles T~, T2.
A s s u m i n g t h a t (a ) is tr u e w e h a v e :
l l ( x L , y L ) = f ( x L ) g ( y L ) ,
l l ( x L , y U ) = f ( x L ) g ( y V ) , a n d
l l ( x U , y L ) -~ - f ( x U ) g ( y L ) .
T h i s i m p l i e s t h a t o n e c a n p a r t it i o n t h e o r i g i n a l r e c t a n g l e
i n t o t h e f o l l o w i n g t w o d i s j o i n t t ri a n g l e s ,
T 1 = [ ( x L , y L ) , ( x L , y U ) , ( x U , y L ) ] , a n d
1 2 ( x U , y U ) = f ( x U ) g ( y U ) ,
1 2 ( x L , y U ) = f ( x L ) g ( y U ) ,
1 2 ( x v , y L ) = f ( x V ) g ( y L ) .
a t w h o s e v e r t i c e s t h e l i n e a r f u n c t i o n s l l ( x , y ) , 1 2 ( x , y ) m a t c h t h e o r i g i n a l p r o d u c t
o f u n i v a r ia t e f u n c t i o n s f ( z ) g ( y ) r e s p e c t i v e l y ( S e e F i g u r e 1 ) .
I f l ( z , y ) w e r e n o t t h e c o n v e x e n v e l o p e o f f ( z ) g ( y ) o v e r th e r e c t a n g u l a r d o m a i n
T~ t h e n , t h e r e w o u l d b e a t h i r d c o n v e x f u n c t i o n 13 ( x , y ) u n d e r e s t i m a t i n g f ( x ) g ( y )
o v e r T~ a n d a p o in t ( 2 , if ) E R su c h th a t :
l ( : , ~ 7 ) < 1 3 ( 2 , ~ ).
S u p p o s e th a t ( 2 , y ) E ~ . T h e n ( 2 , ~ ) is a u n i q u e c o n v e x c o m b i n a t i o n o f t h e t h r e ee x t r e m e p o i n t s v 1 v 2 , v 3 o f T 1 . H e n c e , f o r t h e a f f in e f u n c t i o n l , t h e r e e x i s t s u n i q u e
~ i = l A i = 1 s u c h t h atos i t iv e A1, )~2 ,/~3 sa t i s fy in g 3
S O L U T I O N S O F N O N L I N E A R E Q U A T I O N S 153
B e c a u s e 13 i s th e c o n v e x e n v e l o p e o f f ( x ) g ( y ) ins ide T~, ( i ) 13 i s co n ve x a nd ( i i ) i t
m a t c h e s f ( x ) g ( y ) a t a l l v e r t e x p o i n t s l i k e l ( x , y ) d o e s w h i c h i m p l i es :
t 3 ( s : , ~ ) = 1 3 : ~ < : ~ d 3 ( ~ ~ ) = ~ , x d ( r = l ( s : , ~ ) .
i = 1 i = 1
T h i s c o n t r a d i c t s t h e i n i t i a l h y p o th e s i s l (:~, ~7) < / 3 ( x , Y ) a n d t h e r e f o r e , l ( x , y ) is
i n d e e d th e c o n v e x e n v e l o p e o f f ( x ) g ( y ) i n ~ . N o t e t h a t a s im i l a r a r g u m e n t h o l d s
i f (: L 77) E T ~ . M o r e o v e r , d e p e n d i n g o n w h i c h m o n o t o n i c i t y c o m b i n a t i o n ( a ), (b )
, (c ) o r (d ) i s t r u e i t i s a lw a y s p o s s ib l e t o p a r t i t i o n ~ i n to tw o t r i a n g l e s T 1 , T 2
b y h a l v i n g a l o n g o n e o f t h e d i a g o n a l s . T h e r e f o r e , b y f o l l o w i n g t h e s a m e l i n e o f
t h o u g h t f o r c o m b i n a t i o n s ( b ) , ( c ) a n d ( d ) i t i s s t r a i g h t f o r w a r d t o e x t e n t t h i s p r o o f
f o r a l l m o n o t o n i c i t y c o m b i n a t i o n s . [ ]
T h e a n a l y s i s f o r t h e c o n v e x l o w e r b o u n d i n g o f p r o d u c t s o f t w o u n i v a r ia t e f u n c t io n s
c a n b e e x t e n d e d t o a c c o m m o d a t e th e p r o d u c t o f N u n i v a r i a te f u n c ti o n s . T h i s is
a c c o m p l i s h e d b y s u c c e s s iv e l y c o n v e x l o w e r b o u n d i n g p a i rs o f u n i v ar ia t e f u n c ti o n s
i n a r e c u r s i v e m a n n e r u n t i l n o p a i r s a r e l e f t . O n e o f t h e p o s s i b l e a l t e r n a t i v e s o f
c o m b i n i n g p a i r s i s t o s ta rt w i t h c o n v e x l o w e r b o u n d i n g t h e l a st tw o f u n c t i o n s o f th e
p r o d u c t a n d w o r k y o u r w a y t o t h e fr o n t o f t h e e x p r e s si o n . T h e o r e m ( 3) s t a te s t h att h is p r o c e d u r e y i e l d s a c o n v e x l o w e r b o u n d i n g f u n c t i o n f o r t h e i n it ia l p ro d u c t .
T H E O R E M 3 . I f f i E c= [x ,xy I - - , r + , i = 1 , . . . , N a n d
L ( x ) = Y o
w h e r e
= L ( Y j + l f j + l ( X j + l ) ) L La x { + ( f j + l y j+ l ) _ l _ + L - - Y j + l f j + l ,
+ ( f Y - I - l Y J - i - 1 ) " i - + ( Y Y ' - I - l f j - t - I ( Z J " P l ) ) - - Y j ' - P - l f j + l ,U ,
j = 0 , . . . , N - 3
a n d
Y N - 2 m a x { + ( f D _ l f N ( X N ) ) - i - + ( f D f N - I ( X N - 1 ) ) L L
q ~ ( f N U _ l f N ( Z N ) ) " - ' l - ( f U N f N - I ( X N - 1 ) ) - - f N U _ l f N }
in f su p L X L LX N , X N _ I ( r N ) ) - } - ~ ) ( f L f N - I ( X N - 1 ) ) - f N - l f ~ r
t h e n( i ) L ( x ) i s c onv e x , V x E [ x L , x U ] 9
N
(ii) 11 f i ( x i ) ) _ L ( x ) , V x E [ x L , x U ] .i----1
P r o o f . S t a r t i n g f r o m t h e b e g i n n i n g o f t h e r e c u r s i v e d e f i n i t i o n o f L ( x ) ,
Y N - 2 i s a c o n v e x f u n c t i o n o f ( X N - I , X N ) a s th e m a x o f tw o c o n v e x f u n c -t i o n s . F o r t h e s a m e r e a s o n Y N - 3 i s a c o n v e x f u n c t i o n o f ( X N - 2 , Y N - 2 ) o r
o t h e r w i s e o f ( X N - 2 , X N - I , X N ) . B y r e c u r s i v e l y s u b s t i t u t i n g y j i n t o t h e e x p r e s -
s i o n f o r Y j - 1 w e d e d u c e t ha t f o r e v e r y j = 0 , . . . , N - 2 , y j is a c o n v e x f u n c t io n
o f ( x j + l , x j + 2 , . . . , X N ) . T h e r e f o r e , L ( x ) = Y0 is a c o n v e x f u n c t i o n o f
(Xl , x2 , 99 9 X N ) w h i c h p r o v e s p a r t ( ii) o f T h e o r e m ( 3) .
F r o m T h e o r e m ( 1 ) a n d t h e s t a te m e n t o f T h e o r e m ( 3) w e h a v e ,
f N _ t ( X N _ I ) f N ( X N ) ~ > - - f N - l f N 'a x ( r -} - ~ ) ( f L f N - I ( X N - 1 ) ) L L
Y N - 2
a n d
~ + I ( X j + I ) Y j + I > m a x ( r + r ( y L + l f j + I ( X j + I ) ) L L- - Y j + l f j + l ,
= y j ,
j = 0 , . . . , N - 3 .
B y c o m b i n i n g t h e s e l a s t t w o s e ts o f i n e q u a li ti e s w e h a v e ,
N
I I s , ( x , / > _ - -
i=1
w h i c h p r o v e s p a r t ( ii) o f T h e o r e m ( 3) . []
T h e o r e m ( 3 ) d e s c r i b e s o n e p o s s i b le w a y o f re c u r s i v e l y c o m b i n i n g p a i r s o f u n i v a r i-
a t e fu n c t io n s . T h e o r e m ( 4) s t a te s w h o m a n y o f th e s e a l t e rn a t iv e s e q u e n c e s e x i s t
f o r c o n v e x l o w e r b o u n d i n g t h e p r o d u c t o f N u n i v a r i a te f u n c t io n s .
w i t h w h i c h t h e s e p o i n ts a p p e a r i n t h e g r a p h o f t h e u n i v a r i a te f u n c t i o n f ( x ) w h i c h
is:
9 . [ d l - l - d u - u ] i ' " .
T h i s n a t u r a l l y p r o v i d e s a p a r t i ti o n i n g o f t h e i n it ia l i n t e r v a l [ a , b ] i n t o c o n v e x
s u b i n t e r v a l s [d l i , d u i ] a n d c o n c a v e o n es [d u i , d l i ] .
T h e p r o c e d u r e f o r l o ca t in g t h e f i rs t p o i n t w h e r e t h e c o n v e x e n v e l o p e c h a n g e s
r e p r e s e n t a t i o n d e p e n d s o n w h e t h e r f ( x ) i s c o n v e x o r c o n c a v e a t x = a . I f f i s
c o n c a v e a t x = a t h e n t h e in i ti a l s e g m e n t o f th e c o n v e x e n v e l o p e is a l in e . T h e
n e x t s e g m e n t o f t h e c o n v e x e n v e l o p e is t h e f u n c t i o n it s e ff s ta r ti n g a t t h e p o i n t x
w h e r e th e s l o p e f ~ o f f e q u a l s t h e s l o p e o f th e l in e c o n n e c t i n g a w i t h x .
f ( x ) - f ( a ) = f ' ( x ) .
x - - a
N o t e t h a t x b e l o n g s t o o n e o f t h e c o n v e x s u b i n t e r v a l s [ d l i , u i ] s i n c e f m u s t b e
c o n v e x a t x . T h i s i m p l i e s t h a t t h e t a s k o f l o c a t i n g x c o r r e s p o n d s t o d r a w i n g a t a n g e n t
f r o m t h e f i x e d p o i n t ( a , f ( a ) ) t o e a c h o n e o f t h e c o n v e x f u n c t i o n r e p r e s e n t a t i o n s
d e f i n e d i n t h e s u b i n t e r v a l s [ d l i, u i ] . B e c a u s e t h e r e e x i s t s a s i n g l e t a n g e n t t o a
c o n v e x f u n c t i o n d r a w n f r o m a p o i n t o u t s i d e t h e a c o n v e x f u n c t i o n [ 2 1 ] a n y s t a n d a r d
b i s e c t i o n a l g o r i t h m c a n b e u t i l i z e d t o l o c a t e x . T h e c o r r e c t s u b i n t e r v a l [ d l i , u i ] i st h e n t h e o n e w h i c h p r o v i d e s a l i n e t h a t d o e s n o t c u t - o f f a n y p o r t i o n o f t h e c u r v e
f ( x ) .
I f f i s c o n v e x a t x = a , t h e n t h e i ni ti al s e g m e n t o f t h e c o n v e x e n v e l o p e
c a n b e e i t h e r a l i n e o r t h e f u n c t i o n i t se l f. I f t h e r e e x i s ts a c o n v e x s u b i n t e r v a l
[ l i u i ] , = 2 , . . . w h e r e t h e e q u a t i o n f ( ) f ( ) = f ' ( ) ( - a ) h a s a s o l u t i o n
x w h i c h d e f i n e s a l i n e t h a t d o e s n o t c u t - o f f a n y p o r t i o n o f t h e c u r v e f ( x ) t h e n t h e
i n it ia l s e g m e n t o f t h e c o n v e x e n v e l o p e is a l in e c o n n e c t i n g t h e p o i n t s ( a , f ( a ) ) a n d
( x , f ( x ) ) . O t h e r w i s e , t h e i n it ia l s e g m e n t o f t h e c o n v e x e n v e l o p e i s t h e f u n c t i o n
f i ts e lf . T h e l a st p o i n t o f th i s s e g m e n t Xl i s f o u n d b y l o c a t i n g t h e e n d p o i n t s
X l , x 2 o f th e n e x t s u b i n te r v a l w h e r e t h e c o n v e x e n v e l o p e b e c o m e s a l in e s e g m e n t .
T h i s c o r r e s p o n d s t o d r a w i n g a c o m m o n t a n g e n t t o f i n s i d e t h e i n t er v a l s [ d / l , d u l l
a n d [ d l i, d u i ] , i = 2 , . . . . a n d i s t h e s o l u t io n o f t h e fo l l o w i n g s y s t e m o f t w o
e q u a t i o n s :
f ( x 2 ) - f ( z l ) = f ' ( x l ) = : ' ( x 2 ) ,
X 2 - - X l
w h e r e X l E [ d l l , d u l l a n d x 2 E [ d l i , d u l l , i = 2 , . . . . A g a i n , t h e c o r r e c t s u b i n t e r v a l
[ d l i , d u i ] , i = 2 , . . . i s t h e n th e o n e f o r w h i c h t h e l in e c o n n e c t i n g t h e p o i n ts
( X l , f ( x l ) ) a n d ( x2 , f ( x 2 ) ) d o e s n o t c u t - o f f a n y p o r t io n o f t h e c u r v e f ( x ) . T h e
n e x t l i n e s e g m e n t is t h e n f o u n d b y i te r a ti v e ly s o l v in g t h e s y s t e m o f t w o e q u a t io n s
f o r lo c a t i n g t h e n e w p o i n t s x l , x 2 . T h i s t im e h o w e v e r , X l E [ d l i , d u i ] a n d x 2 E
[ d l j, d u j ] , j = i + 1 , . . . . T h i s is c o n t i n u e d u n t il t h e e n d p o i n t x = b i s m e t . B a s e d o n
t h is a n a l y s i s a n i t e r a t iv e p r o c e d u r e is d e f i n e d f o r c o n s t r u c t i n g t h e c o n v e x e n v e l o p e
160 C . D . M A R A N A S A N D C . A . F L O U D A S
P R O P E R T Y 4 . T h e m ax i m u m sep a ra t io n b e t w een s an d f i s b o u n d e d an d
propor t iona l to a and to the square o f the d iagona l o f the cur ren t box co n-
straints.
m ax ( f ( x ) - Z : ( x ) ) = 8 8 x U - xLl] 2.x L < x < x t r
P R O P E R T Y 5 . T h e u n d e r e s t im a t o r s co n s t ru c t ed o v e r su p e rse ts o f th e cu r r en t
se t a r e a lway s l e s s t i g h t t han the unde res t imator cons t ruc ted over the cu r ren t box
cons t r a in t s fo r ev ery po in t w i th in the cur ren t box cons t ra in ts .
P R O P E R T Y 6 . E co r re sp o n d s to a r e lax ed d u a l b o u n d o f t h e o ri g in a l f u n c t io nf .
This type o f conve x lower boun ding i s u t il i zed fo r a rb it r a ry no nco nve x func t ions
which l ack any spec i f i c s t ruc tu re tha t migh t enab le the cons t ruc t ion o f a more
cu s t o m i zed co n v ex l o w er b o u n d i n g f u n c ti o n .
6 . P r o c e d u r e f o r L o c a t i n g A l l S o l u t i o n s
6.1. DESCRIPTION
A de te rmin i s t ic g loba l op t imiza t ion approach i s p roposed fo r loca t ing a l l e - so lu t ions
of no n l inear sys tems o f equ a l i ti es sub jec t to no n l inear inequa l i ty cons tr a in ts (S ).
By in t roduc ing a s l ack var i ab le , the in i ti a l p rob lem (S) i s tr ansfo rme d in to a g lob a l
o p t im i za t io n p r o b l em ( P ) w h o se m u l ti p le g l o b a l m i n i m a ( i f an y ) co rr e sp o n d t o t h e
mu l t ip le so lu t ions o f (S ). A zero ob jec t ive func t ion va lue d eno tes the ex i s t ence o f
a so lu t ion w hereas a s t ri c tly pos i t ive ob jec t ive func t ion va lue impl ies tha t (S ) has
no so lu tions . Th i s def ines a one- to -one cor respo nden ce be tw een so lu t ions o f the
co n s t r a in ed sy s t em o f eq u a t io n s ( S ) an d m u l t ip l e g l o b a l m i n i m a w i t h an o b j ec ti v eva lue o f ze ro fo r p rob lem (P) . How ever , i t has been show n [14] tha t no a lgor i thm
can ex ac t ly loca te a l l m ul t ip le g loba l min im a of (P ) wi th a f in it e nu m ber o f func-
t ion eva lua tions . A cor ro la ry o f thi s r esu l t [14] i s tha t no a lgor i thm can a lwa ys
loca l i ze , wi th a f in i te num ber o f func t ion ev a lua t ions , a l l g loba l ly op t imal po in t s
b y co m p ac t su b r ec t an g le s i n o n e - t o - o n e co r r e sp o n d en ce w i t h t h em . T h e r e f o re , a
m ore t r ac tab le t a rge t, t han f ind ing a l l exac t g loba l m in im a of (P ) , i s to f ind a rb i-
t r ar i ly smal l d i s jo in t subrec tang les con ta in ing a l l g loba l ly op t im al po in t s o f (P ) ,
poss ib ly no t in a one- to -one cor responden ce .
These mul t ip le e -g loba l min ima of (P ) , ( i f any) can then be loca l i zed based
o n a b r an ch an d b o u n d p r o ced u r e i n v o l v in g t h e su cces s i v e r e f in em en t o f co n v ex
re laxa t ions (R) o f the in i ti a l p rob lem (P). Fo rmu la t ion (R) i s ob ta ined by r ep lac ing
th e n o n c o n v e x fu n c ti o n s '~3fin'~ --hn'~176 vk"n~ w it h t i g h t , c o n v e x l o w e r b o u n d i n g
func t ions ~ n o n c ~znone p,none by fo l lowing some of the t echn iques d i scussed in+ , J ~ - - , 3 ~ k '
t he p rev ious sec t ion . Because (R) i s convex , i t s g loba l min imum wi th in some
S T E P 0 - I n i t i a l i z a t i o nA s i z e t o l e r a n c e e , a n d a f e a s i b i l i ty to l e r a n c e e a r e s e l e c t e d a n d t h e i te r a t i o n
c o u n t e r I t e r i s s e t t o o n e . A p p r o p r i a t e g l o b a l b o u n J s X L B D ~ X U B D o n x a r e c h o s e n
a n d l o c a l b o u n d s X L ' I t e r , X U ' I t e r f o r t h e f i rs t i t e r a ti o n a r e s e t t o b e e q u a l t o t h e g l o b a l
o n e s . F i n a l l y , s e l e c t a n i n i t i a l p o i n t X c ' I t e r t h a t s a t i s f i e s t h e l i n e a r e q u a l i t i e s a n d
c o n v e x i n e q u a l it i e s o f ( P ) .
S T E P 1 - F e a s i b ~ i ty a n d C o n v e r g e n c e C h e c k
I f t h e m a x i m u m v i ~ a t i o n o f a ll n o n c o n v e x c o n s tr a in t s o f ( P ) c a l c u la t e d a t t h e
c u r r e n t p o i n t x c' I te~ f o r ( s = 0 ) i s l e s s t h a n e ,
m a x [ m a x h~ ~ max g ~ ~[ j~H.o .r ' keX.o.oi --
t h e n t h e p o i n t x r i s a e y - s o l u ti o n o f (S ) . F a t h o m c u r r e n t r e c t a n g l e i f it s d i a g o n a l
i s le s s t h a n e ~,
I Ix U , I t e r _ x L , I t e r l l < ,~ .
a n d G O T O S t e p 4 . O t h e rw i s e , c o n t i n u e w i t h S T E P 2 .
S T E P 2 - P a r t i ti o n i n g o f C u r r e n t R e c t a n g l e
Ix L , I t er , x U , It e~] i s p a r t i t i o n e d i n t o t h e f o l l o w i n g t w oT h e c u r r e n t r e c t a n g l e
r e c t a n g l e s ( r = 1 , 2 ) :
" L , I t e rX 1
L , I t e rX l I t e r
xU1 , I t e r
L , i t e r _ _ x U , I t e r , ~X l l t e r T l i f e r
2
L , I t e r U , I t e rX N X N
L , I t e r x U l , I t e rX 1
L ' I t e v - - x U ' I t e r ~X l i t e r " t- l i f e r ) U , t e r
2 X I I t e r
:
X ~ I t e r X ~ I t e r
w h e r e l I t e r c o r r e s p o n d s t o t h e v a r i a b l e w i t h t h e l o n g e s t s i d e i n t h e i n i t i a l
r e c t a n g l e ,
l i t er = a r g m a x ( x ~ , z te r _ z L , I t e r )
S T E P 3 - S o l u ti o n o f C o n v e x P r o b l e m s I n s i d e S u b r e c t a n g l e s
S o l v e t h e f o l lo w i n g c o n v e x o p t i m i z a t io n p r o b l e m ( R ) i n b o t h s u b r e c ta n g l e s
( r = 1 , 2 ) b y u s i n g a n y c o n v e x n o n l i n e a r s o l v e r ( e .g . M I N O S 5 . 4 [2 8 ]) . I f t h e
r , l ter i s n e g a t i v e t h e n , i t i s s t o re d a l o n g w i t h t h e v a l u e o f th e v a r i a b l e s xs o l u t i o n S so
M a t h em a t i ca l p r o o f t h a t t h e p r o p o sed p r o ced u r e is g u a r an t eed to co n v e r g e t o ase t o f d i s jo in t r ec tang les con ta in ing a l l g loba l m in im um so lu tions o f (P ) i s g iven
based on the ana lys i s o f a s t andard de te rmin i s t i c g loba l op t imiza t ion a lgor i thm
p r esen t ed in [1 5]. B ecau se t h e em p l o y ed b r an ch an d b o u n d t ech n i q u e fa t h o m s
only r ec tang les g u a r a n t e e d n o t t o c o n ta i n a n y g l o b a l m i n i m a o f (P) no solut ions
of (P ) which a re a t l eas t e r apar t a r e missed . By fo l lowing the p roof in [23] , a
suf f ic i en t cond i t ion fo r the p roposed b ran ch and bou nd a lgor i thm to be co nve rgen t
to the g loba l min ima , r equ i r es tha t the bound ing opera t ion must be c ons i s t e n t an d
the s e lec t ion opera t ion b o u n d i m p r o v i n g .
A boun ding opera t ion i s ca l led c o n s i s t e n t i f ( i) a t every s t ep any un fa thom ed
par t i t ion can be fu r ther r e f ined , and ( i i ) fo r any in f in i t e ly decreas ing sequence
o f su cces s i v e ly r e fi n ed p a r ti ti o n e l em en t s t h e g ap b e t w e en t h e l o w er an d u p p e r
bou nds g oes to ze ro as the i terat ions go to inf in ity. D ue to prop er t ies (1) , (2) , (3) o f
S ec t i o n 2 t h e g ap b e t w een t h e l o w e r an d u p p e r b o u n d f o r an y p a rt it io n e l em en t
goes to ze ro as the s i ze o f the par t it ion e lem ent goes to ze ro as w el l . Fur therm ore ,
the em ploy ed b i sec t ion sub d iv i s ion p rocess (b i sec t ion a long the longe s t s ide) is
exhaus t ive beca use the s i ze o f an in f in i te ly par t i tioned e le m ent goes to ze ro . There-
fo re , the bou nding opera t ion i s cons i s ten t . Al so , the em ploy ed se lec t ion opera t ion
is b o u n d i m p r o vi n g b ecau se t h e p a r t it io n e l em en t w h e r e t h e ac t u a l l o w er b o u n d
i s a t t a ined i s se lec ted fo r fu r ther par t it ion in the im m edia te ly fo l lowing i te r a tion .
Therefore acco rd ing to T heo rem IV.3 . in [15] the em ploy ed g loba l op t imiza t ion
a lgor i thm i s c o n v e r g e n t t o the g loba l min im a of (P ). In the nex t sec t ion the p ropose d
g loba l op t im iza t ion a lgor i thm i s app l i ed to a nu m be r o f exam ple p rob lems.
7. Com putational Results
In th is sec t ion , a num ber o f t es t p rob lem s a re add ressed wh ich a re a ime d a t de te r-m in ing the ab i l ity o f the appro ach to f ind a ll so lu t ions o f cons t r a ined sys tems o f
eq u a t io n s w i t h r ea so n ab l e co m p u t a t i o n a l r eq u ir em en t s . T h e p r o p o sed b r an ch an d
b o u n d co n v ex l o w er b o u n d i n g a lg o r i th m h as b een i m p l em en t ed i n G A M S [5 ] an d
com puta t iona l t imes a re repor ted fo r a ll exam ples on a HP -730 wo rks ta t ion w i th
s ize and f eas ib i l ity to le rances o f 10 -4 .
EX A M PL E 1 . The fir st exam ple invo lves the loca t ion o f a l l t he s t a tionary po in t s
o f the Him m elb lau func t ion as descr ibed in [33] .
U s i n g t h e e x p o n e n t i a l v a r i a b l e t r a n s f o r m a t i o n d e s c r i b e d i n S e c t i o n 4 , t h e s i n g le
s o l u t i o n o f t h e p r o b l e m i s f o u n d a f t e r 6 3 1 i t e r a t io n s a n d 3 1 . 7 s e c o n d s o f C P U t i m e(see Tab le I I I ).
w hich w as m issed in a l l p rev ious a t t empts a t so lv ing th i s p rob lem.
8 . S u m m a r y a n d C o n c l u s io n s
I n t h is p ap e r a d e t e r m i n is t ic b r an ch an d b o u n d t y p e a l g o r it h m w as p r o p o sed f o r
loca t ing a ll e -g loba l so lu t ions o f ce r t a in c l asses o f cons t r a ined sys tem s o f non l inear
eq u a ti o n s. T h e ap p r o ach i s b a sed o n t h e o n e - t o -o n e co r r e sp o n d en ce b e t w een t h e
m u l t i p le so l u ti o n s o f t h e n o n l i n ea r sy s t em s an d t h e m u l t i p le g l o b a l m i n i m a w i t h a
zero ob jec t ive va lue fo r the r esu l t ing no nco nve x op t im iza t ion p rob lem. Al l m ul -
t ip l e e - g l o b al m i n i m a o f t h e n o n co n v ex o p t i m i za ti o n p r o b l em a r e l o ca l iz ed b a sed
o n a co n s t r u c t i o n o f u p p e r b o u n d s w i t h f u n c t i o n ev a l u a t i o n s an d l o w er b o u n d
o n t h e g l o b a l m i n i m u m so l ut io n t h r o u g h t h e co n v ex r e l ax a ti o n o f t h e co n s t r a in t
s e t an d t h e so l u ti o n o f co n v ex m i n i m i za t io n p ro b l em s . B ased o n t h e f o r m o f t h e
par t ic ipa ting func t ions , a nu m ber o f a l t e rna tive t echn iques fo r co ns t ruc t ing th i scon vex r e laxa t ion a re p roposed . In par t i cu la r , by t ak ing advan tage o f the p roper t i es
o f p r o d u c t s o f u n i v a r ia t e f u n c ti o n s , cu s to m i zed co n v ex l o w er b o u n d i n g f u n c ti o n s
are in t roduced fo r a l a rge nu m ber o f express ions tha t a re o r can be t r ansfo rmed
in to p roduc t s o f un ivar i a te func t ions . The u t il i ty o f these con vex low er bound ing
func t ions tr anscends the spec i fi cs o f the roo t f ind ing p rob lem beca use they can be
i n co r p o ra t ed i n an y co n v ex l o w er b o u n d i n g a l g or it h m . A l t e r n a ti v e co n v ex r e l ax -
a t io n p r o ced u r e s i n v o l v e e i t h e r th e d i f fe r en ce o f t w o co n v ex f u n c t i o n s em p l o y ed
in o~BB [23] o r the expon en t i a l var iab le t r ansfo rmat ion ba sed underes t im ator s
em p l o y ed f o r g en e r a l iz ed g eo m e t r ic p r o g r am m i n g p r o b l em s [2 4]. T h e p r o p o sed
branch and bo und approac h i s guaran teed to loca l i ze a l l e -so lu tions o f (S ) wi th in
arb it ra r ily sm al l r ec tang les in a f in i te num ber o f i te r a tions . A n um be r o f exam -
p l e p r o b lem s f r o m m an y a r ea s o f re sea r ch h av e b een ad d r e s sed an d i n a l l ca se s ,
co n v e r g en ce to a ll m u l t i p le so lu t io n s w as ach i ev ed w i t h r ea so n ab l e co m p u t a t io n a l
e f fo r t. Fu r thermore , in ce r t a in cases ne w so lu tions we re iden t if i ed .
A p p e n d i x AC o n v e x L o w e r B o u n d i n g E x a m p l e s o f U n i v a r ia t e F u n c t i o n s
I n t hi s ap p en d i x a n u m b er o f co n v ex l o w er b o u n d i n g s it u at io n s a r e ex am i n ed .
( 1 ) B i l in e a r t e r m s
T h e co n v ex u n d e r e s ti m a t i o n o f b i l in ea r te r m s x y i ns ide the r ec tangu la r r eg ion
[ x L , x U ] x [ y L ,y U ] c a n b e h an d l ed b y i n v o k i n g T h e o r em ( 1 ) an d se tt in g f ( x ) = x
Note tha t , t he lower bounding procedure can be appl i ed to a nega t ive ly-s igned
bi l inear term - x y by set t ing f ( x ) = - x and g ( y ) = y :
- x y > _ m ax { - x V y - y L x + x U y L ,
- -xL y - - y V x + x L y U } .
B ecause in t h is ca se f ( x ) , g ( y ) a r e l i near and therefore conc ave func t ions , we havef rom T heo rem (2) tha t t he ob ta ined convex low er bou nding func t ions a re iden t ica l
to the co nvex en ve lop es as we re f ir st der ived by [3] .
( 2 ) F r a c t i o n a l t e r m s
Co nve x lower bou nding o f the l inear frac tiona l te rm x / g i ns ide the rec tangular
reg ion [ x L , x U J x [yL , yU ] can a l so be accom pl i shed based on Theo rem (1 )b y
select ing f ( x ) = x a n d g ( y ) - 1 .
x > m a x { r 1 6 2
Y
r x
N ote that, r =xy-r, r x = -tr an d
rf i ( Y ~ + v ~ - g i f z s < o '
yLyU
r = x~b(Y L+Y ~- v) i f x s < O .yLyU
Therefore ,
7 - 7 -> m a x -V + i f x L > 0-- x ~ x LY y-~- - -k -~7~ i fx L < 0
A f t e r c o m b i n i n g a l l t h r e e c o n v e x l o w e r b o u n d i n g a l te m a t i v e s , w e o b t a i n t h e f o l-
l o w i n g e i g h t c o n v e x f u n c t i o n s i n x , y , z w h o s e m a x i m u m is a ti g h t c o n v e x l o w e rb o u n d i n g f u n c t i o n fo r xyz:
~x y L xLy xLy L xLy Lx y > m a x + + - - 2 ~ ,
z - [ - i v - - - ; v -
xy L xLy xLy U xLy U xLy L
z U q - " ~ ' 4 - - - Z Z n Z U '
xy U xVy xUy L xVy L xVy v
z L + - - F + - - z z v z L 'xy U xUy xLy U xLy U xUy U
z U -'[- - ' ~ " l- - - Z Z U Z L '
xy L xLy xUy L xUy L xLy L
Z-- -~ "{" ~ "{- - - Z Z L Z v '
xy U xUy xLy U xLy g xUy U
z v + - - Z - + - - Z Z U Z L
xy L xLy xUy L xUy L xLy L
Zu -[" - - ~ "4- --Z ZL ZU 'xy u xVy xUy U xUy U ]
~ L + - -; z- + - - - - ; - - 2 - - 7 ~ ~ 9
A p p e n d i x B
C o n v e x i t y /C o n c a v i t y I d e n t i f ic a t io n o f G e n e r a l i z e d P o l y n o m i a l T e r m s
I n t h i s a ppend ix , neces s a ry a nd s u f f i c i en t co nd i t io ns a re pro v ided f o r co n v ex i -
t y /c o n c a v i ty o f g e n e r a l iz e d p o l y n o m i a l t e rm s o f t h e fo rm :
z d li , d iEN, i= l , . . . ,N .
G e n e r a l i z e d p o l y n o m i a l t e rm s a r e a s p e c ia l c a s e o f p r o d u c t s o f u n i v a r ia t e f u n c t io n s
b y s e l e c ti n g fi(xi) = x~~.Fi r s t , t he tw o va r i ab l e c a se f = x'~yb i s c o n s i d e r e d .
F i n a l ly , it w i l l b e s h o w n t h a t t h e s e r e q u i r e m e n t s c a n n o t b e s a ti s fi e d i f m o r e t h a n
o n e d i , i = 1 , . . . , N i s p o s i t i v e . L e t d i ~ , d i 2 > 0 , t h e n b e c a u s e ( i l , i : ) E P l w e
h a v e : -
d i l ( 1 - d , ~ ) < 0 a n d d , , ( 1 - d ~ ) < 0 .
H o w e v e r , d il , d i2 > O , t h e r e f o r e
d i a _ > l a n d d i2 _ > 1 .
F u r t h e r m o r e , ( i l , i 2) E P 2 s o
(d i~ d i : ) (1 - d i l - d i 2 ) ~ 0
o r
( 1 - d i l - d i 2 ) > _ 0
w h i c h is in c o n t r a d i c t i o n w i t h d i l , d ! ~ _> 1 . T h e r e f o r e , a s s u m i n g c o n d i t i o n s ( i ) o r( ii ) t h e n f i s c o n v e x i n f o r a l l x E ~ + .
B y f o l l o w i n g t h e s a m e l i n e o f t h o u g h t , f i s c o n c a v e i f a l l t e r m s C N - k ( d , x ) / ~ N - k ,
k = 1 , . . . , N m a i n t a i n c o n s t a n t s ig n f o r e v e r y A > 0 a n d f o r e v e r y x E ~ N . T h i s
i s tru e i f ,
V k = I . . . , N , C N - k ( d , k ) > 0 , V x E ~ N .
T h i s c a n b e w r i t t e n e q u i v a l e n t l y a s ,
T h e r e f o r e , i f c o n d i t i o n ( i i i) i s t r u e t h e n f i s c o n c a v e i n f o r a l l x E N +N . [ ]
A c k n o w l e d g e m e n t s
F i n a n c i a l s u p p o r t f r o m t h e N a t i o n a l S c i e n c e F o u n d a t i o n u n d e r G r an ts C B T - 8 8 5 7 0 1 3 ,
C T S - 9 2 2 1 4 1 1 , A i r F o r c e O f f ic e o f S c ie n t if ic R e s e a rc h , a s w e l l a s E x x o n C o . , A m o -
c o C h e m i c a ls C o . , M o b i l C o . , a n d T e n n e s s e e E a s tm a n C o . i s g ra t ef u ll y a c k n o w l -
e d g e d .
R e f e r e n c e s
1 . R . Hors t , P . M. Pa rda los and N . V . Thoa i , Introduction to Global O ptimization. K l u w e r A c a d e m i c
Publ ishers , 1995 .
2 . R . H o r s t a n d E M , P a rd a lo s . Handb ook o f Global Optimization. K l u w e r A c a d e m i c P u b l is h e r s,
1995.
3 . E A . A 1 -K h a y y a l a n d J . E . F a lk . J o in t l y c o n s tr a in e d b i c o n v e x p ro g ra mm in g . Math. Opers. Res.,8:523, 1983.
4 . I . E A n d ro u la k i s , C . D . , M a ra n a s , a n d C . A . F lo u d a s . c ~B B : G lo b a l O p t imiz a t io n fo r C o n s t r a in e d
N o n c o n v e x P r o b l e m s . J. Global Opt., 7(4) , 1995 ( fo r thcoming) .5 . A . B ro o k e , D . K e n d r i c k a n d A . M e e ra u s . GAMS: A User's Guide. S c ie n t i fi c P r e s s , P a lo A l to ,
CA. , 1988 .6 . L . G . B n l l a rd a n d L . T . B ie g le r . I t e r a t i v e L in e a r P ro g ra m min g S t r a t e g i e s f o r C o n s t r a in e d S imu -
la t ion . Computers chem. Engng., 15(4):239, 1991.
7 . H . S . C h e n a n d M . A . S t a d th e rr . O n s o lv in g l a rg e s p a r s e n o n l in e a r e q u a t io n s y s t e ms . Comp.chem. Engnr., 8:1, 1984.
8 . D . Dav idenko . Dokl. Akad. Nauk USSR, 88:601, 1953.
9 . E . D o e d e l . AUT O: Software fo r Continuation an d Bifurcation Problems in Ordinary DifferentialEquations. A p p l i e d M a th e ma t i c s , C a l t e c h , P a s a d e n a , C A , 1 98 6.
1 0. I . S . D u f f , J . N o c e d a l a n d J . K . R e id . T h e u s e o f l i n e a r p ro g ra m min g fo r t h e s o lu t i o n o f s p a r s e
s e t s o f n o n l in e a r e q u a t io n s . SIAM J. Sci . S tat . Comp., 8:99, 1987.
1 1. G . B . F e r r a ri s a n d E . T ro n c o n i . B U N S L I - a F O R T R A N p ro g ra m fo r s o lu t io n o f s y s t e ms o f
n o n l in e a r a lg e b ra i c e q u a t io n s. Computers chem. Engng, 10:12, 1986.
1 2 . C . B . G a rc i a a n d W . I . Z a n g w i l l . Pathways to Solutions, Fix ed Points a nd Equilibria. P e n t i c e
Hall , 1981.
13 . E . R . Hansen . Global Optimization Using Interval Analysis. M a rc e l D e k k a r , N e w Y o rk , N Y ,
1992.
1 4 . P . H a n s e n , B . J a u ma rd a n d S . H . L u . G lo b a l o p t imiz a t i o n o f u n iv a r ia t e L ip s c h i t z f u n c t io n s : I .
S u rv e y a n d p ro p o e r t i es . Math. Prog., 55:251, 1992.
15 . R . H ors t and H . Tuy . Global O ptimization. Spr inger -V er lag , 1s t. ed i t ion , 1990 .
1 6 . R . B . K e a r fo t t , M . D a w a n d e , K . D u a n d C h . H u . IN T L IB , a P o r t a b l e F O R T R A N 7 7 In t e rv a l
S t a n d a rd F u n c t io n L ib ra ry . in press, 1994.
1 7 . R . B . K e a r fo t t a n d M . N o v o a . IN T B IS , a P o r t a b l e In t e rv a l N e w to n /B i s e c t i o n P a c k a g e . A C MTrans. Math. Soft., 16:152, 1990.
18 . R . W . Klop fens te in . J . Assoc. Comput. Mach., 8:366, 1961.19 . E . Lah aye . C.R. Acad. Sci. Paris, 198:1840, 1934.
20 . J . Ler ay and J . Schander . Ann. Sci . Eco leNo rm Sup., 51:45, 1934.
21 . D . G . Luenberger . Linea r and Nonlinear Programming. A d d i s s o n - W e s l e y ,R e a d in g , M A , 1 98 4 .
2 2 . C . D . M a ra n a s a n d C . A . F lo u d a s . G lo b a l O p t imiz a t io n fo r M o le c u l a r C o n fo rm a t io n P ro b le ms .
Ann. Oper. Res., 42:85, 1993.
2 3 . C . D . M a ra n a s a n d C . A . F lo u d a s . G lo b a l M in imu m P o te n t i a l E n e rg y C o n fo rma t io n s o f S ma l lM o le c u le s . J. Glob. Opt ., 4:135, 1994a.
182 C . D . M A R A N A S A N D C . A . F L O U D A S
2 4 . C . D . M a r a n a s a n d C . A . F lo u d a s . G l o b a l O p t i m i z a t i o n i n G e n e r a l iz e d G e o m e t r i c P o r g r a m m i n g .submitted to Com p. chem. Engnr., 1994c.
2 5 . K . M e i n t j e s a n d A . P . M o r g a n . C h e m i c a l E q u i l i b r iu m S y s t e m s a s N u m e r i c a l T e s t P r o b l m e s . A C M
Transactions on M athematical Software, 16(2): 143, 1990.2 6 . J . J . M o r 6 . T h e L e v e n b e r g - M a r q u a r d t a l g o r i t h m : i m p l e m e n t a t i o n a n d t h e o r y . I n Numerical
Analysis-Proceedings o f the Biennial Conference, Lecture Notes in Mathema tics, n u m b e r 6 3 0,Ber l in , Germany , 1987 . Spr inger -Ver l ag .
27 . A . P . M organ . Solving Polynomial System s Using Co ntinuation fo r Engineering an d ScientificProblems. Pen t i ce -Hal l , I nc . , 1987 .
2 8 . B . A . M u r t a g h a n d M . A . S a u n d e r s . MINOS5.0 Users Guide. S y s t e m s O p t im i z a t io n L a b o r a -
t o ry , D e p t . o f O p e r a t i o n s R e s e a r c h , S t a n f o r d U n i v e r s it y , C A . , 1 9 8 3. A p p e n d i x A : M I N O S 5 . 0 ,T e c h n ic a l R e p o r t S O L 8 3 - 2 0 .
2 9 . A . N e u m a i e r . Interval Methods for Systems o f Equations. C a m b r i d g e U n i v e r s i t y P r e s s , C a m -
b r i d g e , U K , 1 9 90 .
3 0 . J . R . P a l o s c h i a n d J . D . P e r k i n s . A n i m p l e m e n t a t i o n o f q u a s i - N e w t o n m e t h o d s f o r s o l v i n g s e tso f n o n l i n e a r e q u a t i o n s . Comp. chem. Engnr., 12:767, 1988.
31 . M. J . D . Powel l . Numerical Methods for Nonlinear Algebraic Equations. G o r d o n a n d B r e a ch ,L o n d o n , U K , 1 9 7 0 .
3 2 . H . R a t s c h e k a n d J . R o k n e . E x p e r i m e n t s u s i n g i n t e r v a l a n a l y s i s f o r s o l v i n g a c i r c u i t d e s i g n
p r o b l e m . J. G lob. Opt., 3 :501 , 1993 .
3 3 . G . V . R e k l a i t i s a n d K . M . R a g s d e l l . Engineering Optimization. J o h n W i le y & S o n s , N e w Y o r k ,
NY, 1983.
3 4 . L . T . W a t s o n , S . C . B i l l ip s a n d A . P . M o r g a n . AC M Trans. Math. Soft . , 13:281, 1987.