LP Narrowing: A New Strategy for Finding All Solutions of Nonlinear Equations Kiyotaka Yamamura Naoya Tamura Koki Suda Chuo University, Tokyo, Japan.

LP Narrowing: A New Strategy for Finding All Solutions of Nonlinear Equations

Kiyotaka Yamamura Naoya Tamura Koki Suda

Chuo University, Tokyo, Japan

Introduction

ه In this presentation, we discuss the problem of finding all solutions of a system of n nonlinear equations with a separable mapping:

f (x) = 0 (1)

contained in a box D in Rn.

ه Actually, the algorithm proposed in this presentation can be applied to more general systems of nonlinear equations, but we restrict our discussion to the separable systems because the proposed algorithm is especially efficient for such systems.

In this presentation, we do not consider verified numerical computation because we mainly consider the application to large-scale practical engineering problems where it is enough to obtain approximate solutions.

Finding all solutions of nonlinear equations is an important problem which is widely encountered in science and engineering.

Interval Algorithms

ه As a computational method to find all solutions of nonlinear equations, interval analysis based techniques are well-known.

ه Using the interval algorithms, all solutions of (1) contained in D can be found with mathematical certainty.

ه However, the computation time of the interval algori-thms tends to grow exponentially with the dimension n.

ه Therefore, it is necessary to develop a powerful test for nonexistence of a solution in a given box.

LP Test

ه In [1], a powerful computational test was proposed for nonexistence of a solution to the system of nonlinear equations (1) in a given box X.

ه In [2], the LP test was much improved by introducing the dual simplex method, by which the LP test becomes not only powerful but also efficient. In [2], the Krawczyk-Moore algorithm using the LP test succeeded in finding all solutions to systems of nonlinear equations with n = 200.

ه In [3], an improved version of this algorithm is proposed, which succeeded in finding all solutions of systems of nonlinear equations with n = 300.

Purpose of the Study

We propose an efficient algorithm for finding all solutions of nonlinear equations using a new strategy called LP narrowing.

Boxes containing no solution are excluded.

Boxes containing solution are narrowed.

It is shown that the proposed algorithm could find all solutions of systems of 5000 － 50000 nonlinear equations in practical computation time.

Basic Procedure of Interval Algorithms

An n-dimensional interval vector is denoted by

If there is no solution of (1) in X, then we exclude it from further consideration.

If there is a unique solution of (1) in X, then we compute it by some iterative method.

If these conditions are not satisfied, then bisect X to form two new boxes; we then continue the above procedure with one of these boxes, and put the other one on a stack for later consideration.

X

In interval algorithms, the following procedure is performed recursively, beginning with the initial box X = D. (At each level, we analyze the box X.) Geometrically, X is an n-dimensional box.

Thus, we can find all solutions of (1) contained in D with mathematical certainty.

Nonexistence test proposed in [1]

(2)

For the simplicity of notation, and without loss of generality, in this presentation we assume that (1) can be represented as

LP Test

(2)

(3)

Let the interval extension of gi(xi) over [ai , bi] be [ci , di].Then, we introduce yi and put yi = gi(xi).

Now we replace each nonlinear function gi(xi) in (2) by yi, and consider the LP problem (3).

Then, we apply the simplex method to (3).

X ＝（［ a1, b1 ］ , … ,［ an, bn ］）

LP Test Using the Simplex Method

ه By introducing the LP test to the interval algorithms, all solutions of (2) can be found very efficiently.

ه In [1], this algorithm solves a system of nonlinear equations with n = 60 in practical computation time, although the original Krawczyk-Moore algorithm can solve the system only for n ＜ 12.

・ All solutions of (2) that exist in X satisfy the constraints in (3).

・ If the LP problem (3) is not feasible, then we can conclude that there is no solution of (2) in X.

・ The feasibility of (3) can be checked by the simplex method.

(LP test)

LP Test Using the Dual Simplex Method

ه In [2], it is shown that the LP test can be performed with a few iterations (often no iteration) per box by using the dual simplex method.

ه Using this technique, the LP test becomes not only powerful but also efficient.

ه In [2], this improved LP test is introduced to the Krawczyk-Moore algorithm, which could find all solutions of systems of nonlinear equations with n = 200.

ه In [3], an improved version of this algorithm is proposed, which succeeded in finding all solutions of systems of nonlinear equations with n = 300.

Proposed AlgorithmProposed Algorithm

ه The proposed algorithm is an extension of the algorithm in [2], to which the idea of narrowing a box using LP techniques is introduced.

ه If X is not excluded, then we narrow the box so that no solution is lost, which makes the algorithm much more efficient.

ه Now we explain how X is narrowed efficiently by using the LP techniques.

If the feasible region of (4) is If the feasible region of (4) is emptyempty, then we , then we excludeexclude X X from from further consideration.further consideration.

Narrowing from lower sidefrom lower side in xi - direction

x2

x1

First, we apply the dual simplex method to (4) for i = 1. X

（ 4 ）

If the minimum value If the minimum value xx11** is is

greater than greater than aa11, then we , then we prune prune

the lower partthe lower part of of X.X.*11 xx

Narrowing from lower sidefrom lower side in xi - direction

Then, we repeat the similar narrowing procedure in the xi - directions (i > 1), and narrow the box in all coordinate directions from the lower sides.

x2

x1

X

Narrowing from upper sidefrom upper side in xi - direction

Then, we apply the dual simplex method to (5) for i = 1.

（ 5）

x2

x1

X

If the maximum value If the maximum value xx11** is is

less than less than bb11, then we , then we pruneprune

the upper partthe upper part of of X.X.*11 xx

If the feasible region of (5) is If the feasible region of (5) is emptyempty, then we , then we excludeexclude X X from further consideration.from further consideration.

Narrowing from upper sidefrom upper side in xi - direction

x2

x1

XThen, we repeat the similar narrowing procedure in the xi - directions (i > 1), and narrow the box in all coordinate directions from the upper sides.

Such a series of procedures is called LP narrowing.

LP narrowing

ه As the box becomes smaller, the feasible region becomes smaller, which makes the LP narrowing more and more powerful.

ه The LP problem (4) or (5) can be solved efficiently with a few iterations by the dual simplex method.

The LP narrowing is not only powerful but also efficient and narrows a box very effectively.

ه Notice that, in the LP narrowing, we first per-form the narrowing procedure from the lower sides of all xi-directions, and then perform the narrowing procedure from the upper sides.

ه This is because it makes the number of iteration in the dual simplex method small and makes the algorithm efficient.

Implementation of the Proposed Algorithm

1http://www.gnu.org/software/glpk/

The proposed algorithm can be easily implemented by using the free package GLPK (GNU Linear Programming Kit )1.

ه Callable library for C

ه Intended to solve large-scale LP problems

ه Known to be very efficient; in many cases, it is faster and more robust than lp_solve 5.5.

ه Work in progress and presently under continual development

ه As of the current version 4.27, it is able to handle problems with up to 100 000 constraints.

Advantages of Using GLPK

GLPK is not only very efficient but also well-suited to the proposed algorithm.

ه Since the bounded-variable technique is implemented in GLPK, it can solve the LP problems of the form (4) or (5) very efficiently.

ه We can easily perform the dual simplex method starting from a previously obtained dual feasible basis by using the control parameter “GLP_DUALP”.

Numerical Examples

ه Programming language: C (double precision)

ه Dell Precision T7400 (Intel Xeon 3.4GHz)

ه We used GLPK for solving the LP problems.

ه We compare the computation time of the proposed algorithm and the algorithm proposed in [3].

Example 1

A system of n nonlinear equations

(known as Yamamura1)

n

iiiii niixxxx

1

23 , ,2 ,1 ,08.115.105.2 　

Initial Region:

3000for 20,20,, 20,20

3000for 10,10,, 10,10

nD

nDT

T

Comparison of computation time (s) in Example 1

nn SS Ref.[3]Ref.[3] ProposedProposed

100100 99 4949 0.30.3

200200 1313 11 259259 22

300300 1111 99 345345 99

400400 99 36 85436 854 2222

500500 1313 90 05590 055 5454

:: :: :: ::

10001000 1717 ∞∞ 638638

20002000 99 ∞∞ 5 1135 113

30003000 2727 ∞∞ 37 26137 261

40004000 2121 ∞∞ 7373 950950

50005000 1515 ∞∞ 124124 507507

10 minutes

34 hours

S denotes the number of solutions obtained by the algorithms.

∞ denotes that it could not be computed in practical computation time.

nn SS BoxesBoxes ProposedProposed

100100 99 3939 0.30.3

200200 1313 4141 22

300300 1111 2727 99

400400 99 1919 2222

500500 1313 4747 5454

:: :: :: ::

10001000 1717 5353 638638

20002000 99 1919 5 1135 113

30003000 2727 8989 37 26137 261

40004000 2121 4545 7373 950950

50005000 1515 3737 124124 507507

“Boxes” denotes the number of analyzed boxes of the proposed algorithm.

It is also seen that the number of analyzed boxes is very small in the proposed algorithm, which implies that the LP narrowing is very powerful.

Notice that the number of analyzed boxes does not become large as n increases; it depends mainly on the number of solutions.

Example 2

niixn

xn

jji ,,2 ,1 ,0

2

1

1

3

TD 5.2,5.2,, 5.2,5.2

A system of n nonlinear equations

Initial Region:

Comparison of computation time (s) in Example 2

nn SS Ref.[3]Ref.[3] ProposedProposed

100100 33 8181 0.30.3

200200 33 22 139139 44

300300 33 1212 451451 2020

400400 33 7272 540540 6060

500500 33 118118 409409 123123

:: :: :: ::

10001000 33 ∞∞ 11 086086

20002000 33 ∞∞ 1010 454454

30003000 33 ∞∞ 35 36835 368

40004000 33 ∞∞ 8585 893893

50005000 33 ∞∞ 149149 685685

The total number of pivotings

42,445

The average number of pivotings in solving the LP problem

0.47

It is seen that a similar result is obtained as that in Example 1.Considering the size of the problem, this number is small.

Example 3

nixhxxx iiii ,,2 ,1 ,0exp2 211

TD 5,0,, 5,0

Initial Region:

A system of n nonlinear equations

This system comes from a nonlinear two-point boundary value problem termed the Bratu problem.

nn SS Ref.[3]Ref.[3] ProposedProposed

5050 22 6363 0.020.02

100100 22 77 143143 0.040.04

150150 22 142142 749749 0.080.08

:: :: :: ::

10001000 22 ∞∞ 33

50005000 22 ∞∞ 9999

1000010000 22 ∞∞ 604604

2000020000 22 ∞∞ 3 6983 698

3000030000 22 ∞∞ 88 705705

4000040000 22 ∞∞ 1717 205205

5000050000 22 ∞∞ 2828 296296

Comparison of computation time (s) in Example 3

The total number of pivotings

117,326

The average number of pivotings in solving the LP problem

1.9

The number of solutions is two for all n.

nn SS BoxesBoxes ProposedProposed

5050 22 33 0.020.02

100100 22 33 0.040.04

150150 22 33 0.080.08

:: :: :: ::

10001000 22 33 33

50005000 22 33 9999

1000010000 22 33 604604

2000020000 22 33 3 6983 698

3000030000 22 33 88 705705

4000040000 22 33 1717 205205

5000050000 22 33 2828 2962968 hours

10 minutes

The number of analyzed boxes of the proposed algorithm is only three for all n .

Proposed Algorithm (Example 3)

ه The average narrowing rate per direction (n = 10000)

ه in the first box : 0.590.59ه from the second box : 0.0000720.000072

It is seen that the LP narrowing is very powerful and narrows a box very rapidly, especially when the box contains one solution.

ه The number of analyzed boxes is only three for all n.

Proposed Algorithm (Example 3)x2

x1

Proposed AlgorithmProposed Algorithm

The proposed algorithm narrowed the boxes as above.

This is the reason why the proposed algorithm is very efficient for this problem, and could solve the NP-hardproblem for n = 50000 in practical computation time.

Example 4Example 4 　　 Transistor CircuitsTransistor Circuits

0.01 s

3 solutions

0.02 s

9 solutions

0.10 s

11 solutions

0.07 s

one solution

Systems of nonlinear equations containing many strongly nonlinear terms of the form exp(40xi - 1)

It is seen that all solutions were found in little computation time.

RealPaver [4]

ه RealPaver is a well-known interval software package for solving numerical constraint satisfaction problems including finding all solutions of nonlinear equations.

ه Compare with four algorithms in RealPaver (called BC3N, BC5, weak3B, and 3B) which are considered to be the most efficient algorithms there.

[4]

nn SS BC3NBC3N BC5BC5 weak3Bweak3B 3B3B ProposedProposed

66 55 0.570.57 0.440.44 0.770.77 134134 0.0050.005

88 77 1515 1010 1111 11 390390 0.0090.009

1010 99 250250 174174 201201 5 2695 269 0.0090.009

1212 99 44 057057 2 9642 964 44 580580 6060 428428 0.0110.011

1414 55 106106 913913 7373 844844 149149 005005 ∞ ∞ 0.0130.013

nn BC3NBC3N BC5BC5 weak3Bweak3B 3B3B ProposedProposed

44 0.070.07 0.030.03 0.050.05 1818 0.0060.006

66 11 0.590.59 11 122122 0.0070.007

88 3333 1010 2323 11 423423 0.0080.008

1010 709709 219219 487487 2323 877877 0.0090.009

1212 13 68013 680 3 7353 735 8 6878 687 ∞ ∞ 0.0090.009

nn BC3NBC3N BC5BC5 weak3Bweak3B 3B3B ProposedProposed

1010 0.160.16 0.090.09 0.230.23 2525 0.0090.009

2020 1010 33 77 302302 0.0100.010

3030 349349 5656 123123 1 3401 340 0.0110.011

4040 88 826826 737737 1 8401 840 3 7703 770 0.0120.012

5050 204204 736736 88 397397 7070 529529 88 900900 0.0140.014

Example 1. (sec)

Example 2. (sec) Example 3. (sec)

Comparison with RealPaver

For these problems, the proposed algorithm is much more efficient than the algorithms in RealPaver.

Conclusion

ه An efficient algorithm has been proposed for finding all solutions of separable systems of nonlinear equations using a new strategy called LP narrowing.

ه It has been shown that the proposed algorithm is very efficient and has the possibility of solving large-scale systems of nonlinear equations in practical computation time.

ه The proposed algorithm can be easily implemented by using GLPK.

The proposed algorithm seems to be a useful tool for finding all solutions of nonlinear equations.

ه The interesting feature of this algorithm is that the number of analyzed boxes is very small, although LP problems have to be solved 2n times for each box. The computational cost of solving LP problems 2n times seems to be very large, but actually they can be solved efficiently by the dual simplex method.