Algebraic Geodesy and Geoinformatics - 2009 - PART I METHODS 8 Extended Newton-Raphson Method 8- 1 Introduction In this chapter a special numerical method is introduced, which can solve overdetermined or underdetermined systems directly. Although this method is a local method, it is robust enough to handle also determined systems when the Jacobian is ill- conditioned. Our problem to solve a set of nonlinear equations f (x) = 0 where f: ´ m fi ´ n , namely x ˛ ´ n and f ˛´ m . If n = m and the Jacobi matrix has a full rank everywhere, in other words the system of equations is regular, and in addi- tion, if the initial value of the iteration is close enough to the solution, the Newton-Raphson method ensures quadratic convergence. If one of these conditions fails, for example the system is over or under-determined, or the Jacobi matrix is singular, one can use Extended Newton-Raphson method. Let us consider some examples to illustrate these problems. 8- 2 Overdetermined system 8- 2- 1 Overdetermined polynomial system Let us consider a simple monomial system, Clear@"Global‘*"D f 1 = x 2 ; f 2 = x y; f 3 = y 2 ; This system is a "monomial ideal" and trivial for computer algebra. Ideal = GroebnerBasis @8f 1 ,f 2 ,f 3 <, 8x, y<D 9y 2 , x y, x 2 = That is why the solution is, Off@Solve::"svars"D
22
Embed
Extended Newton-Raphson Method - Extra Materialsextras.springer.com/.../PDFVersion/ExtendedNewton_08.pdfAlgebraic Geodesy and Geoinformatics - 2009 - PART I METHODS 8 Extended Newton-Raphson
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Algebraic Geodesy and Geoinformatics - 2009 - PART I METHODS
8 Extended Newton-Raphson Method
8- 1 Introduction
In this chapter a special numerical method is introduced, which can solve overdetermined or underdetermined systemsdirectly. Although this method is a local method, it is robust enough to handle also determined systems when the Jacobian isill- conditioned.Our problem to solve a set of nonlinear equations
f (x) = 0
where f : Âm
® Ân , namely x Î Ân
and f Î Âm.
If n = m and the Jacobi matrix has a full rank everywhere, in other words the system of equations is regular, and in addi-tion, if the initial value of the iteration is close enough to the solution, the Newton-Raphson method ensures quadraticconvergence.If one of these conditions fails, for example the system is over or under-determined, or the Jacobi matrix is singular, one canuse Extended Newton-Raphson method. Let us consider some examples to illustrate these problems.
8- 2 Overdetermined system
8- 2- 1 Overdetermined polynomial system
Let us consider a simple monomial system,
Clear@"Global‘*"D
f1 = x2;
f2 = x y;
f3 = y2;
This system is a "monomial ideal" and trivial for computer algebra.
We can see that the origin is an isolated singular root with multiplicity of 3. Global polynomial solver NSolve using
numerical Groebner basis can also solve this system,
NSolve@8f1, f2, f3<, 8x, y<D
88x ® 0., y ® 0.<, 8x ® 0., y ® 0.<, 8x ® 0., y ® 0.<<
However, standard Newton - Raphson method built in as FindRoot fails, while the system is overdetermined,
FindRoot@8f1, f2, f3<, 88x, 0.1<, 8y, 0.1<<D
FindRoot::nveq:
The number of equations does not match the number of variables in FindRoot@8f1, f2, f3<, 88x, 0.1<, 8y, 0.1<<D. �
FindRoot@8f1, f2, f3<, 88x, 0.1<, 8y, 0.1<<D
Although we can transform the overdetermined system into a determined one in sense of least squares (see ALESS). Theobjective function is the sum of the square of the residium of the equations,
obj = f12 + f2
2 + f32
x4 + x2 y2 + y4
Considering the necessary conditions for the minimum,
eqx = D@obj, xD
4 x3 + 2 x y2
eqy = D@obj, yD
2 x2 y + 4 y3
Then using the Newton - Raphson method, we get,
FindRoot@8eqx, eqy<, 88x, 0.1<, 8y, 0.1<<D
9x ® 2.03485 ´ 10-8, y ® 2.03485 ´ 10-8=
The convergence is slow and the accuracy of the solution is poor, even changing initial guess, because of the multiple rootexists, see e.g. Chapra and Canale (1998).
Encountered a singular Jacobian at the point 8p1, p2, p3< = 80.182973, 82959.2, -82955.5<. Try perturbing
the initial pointHsL. �
8p1 ® 0.182973, p2 ® 82 959.2, p3 ® -82 955.5<
Global polynomial solver with numerical Groebner basis provides solution, but besides the correct solution in sense of leastsquare, we have other real solutions. However only one positive real solution appears,
u ® -0.18095 - 3.93454 ä, v ® -0.142206 + 19.5258 ä, w ® -0.0213961 - 0.0019722 ä<,8x ® 1.49769 + 3.57562 ä, y ® -0.581399 - 20.8976 ä, u ® -0.18095 + 3.93454 ä,
v ® -0.142206 - 19.5258 ä, w ® -0.0213961 + 0.0019722 ä<,8x ® 1.49453 - 3.52805 ä, y ® -0.565334 + 20.6894 ä, u ® -0.187746 - 3.90416 ä,
v ® -0.106234 + 19.3113 ä, w ® -0.000142429 - 0.0258908 ä<,8x ® 1.49453 + 3.52805 ä, y ® -0.565334 - 20.6894 ä, u ® -0.187746 + 3.90416 ä,
v ® -0.106234 - 19.3113 ä, w ® -0.000142429 + 0.0258908 ä<,8x ® -2.50221, y ® 16.2682, u ® -2.1049, v ® 15.3505, w ® 0.0752342<,8x ® -2.40824, y ® 15.8046, u ® -2.02594, v ® 14.8806, w ® 0.0336009<,8x ® 1.29261 - 0.616946 ä, y ® 0.93699 + 1.44575 ä, u ® -0.225987 + 0.364707 ä,
v ® -0.215959 + 0.152772 ä, w ® -1.54306 - 1.09158 ä<,8x ® 1.29261 + 0.616946 ä, y ® 0.93699 - 1.44575 ä, u ® -0.225987 - 0.364707 ä,
v ® -0.215959 - 0.152772 ä, w ® -1.54306 + 1.09158 ä<,8x ® 0.858909 - 0.627264 ä, y ® 1.52751 + 0.924419 ä, u ® 0.056409 + 0.442294 ä,
v ® 0.152819 + 0.300048 ä, w ® -1.62301 - 0.420833 ä<,8x ® 0.858909 + 0.627264 ä, y ® 1.52751 - 0.924419 ä, u ® 0.056409 - 0.442294 ä,
v ® 0.152819 - 0.300048 ä, w ® -1.62301 + 0.420833 ä<,8x ® 1.48324 + 0.0374272 ä, y ® -0.157492 + 1.55276 ä, u ® -0.328183 - 0.331191 ä,
v ® 0.279366 + 0.0242998 ä, w ® -0.0139936 - 1.53327 ä<,8x ® 1.48324 - 0.0374272 ä, y ® -0.157492 - 1.55276 ä, u ® -0.328183 + 0.331191 ä,
v ® 0.279366 - 0.0242998 ä, w ® -0.0139936 + 1.53327 ä<,8x ® 1.41268 + 0.137626 ä, y ® -0.698037 + 1.6082 ä, u ® -0.117652 - 0.516323 ä,
v ® 0.103892 + 0.360263 ä, w ® 0.369503 - 1.28132 ä<,8x ® 1.41268 - 0.137626 ä, y ® -0.698037 - 1.6082 ä, u ® -0.117652 + 0.516323 ä,
v ® 0.103892 - 0.360263 ä, w ® 0.369503 + 1.28132 ä<,8x ® 0.156897 - 0.646161 ä, y ® 3.26393 + 3.27213 ä, u ® 0.0655816 - 0.565126 ä,
v ® 2.26479 + 3.26473 ä, w ® -1.00249 + 0.23585 ä<,8x ® 0.156897 + 0.646161 ä, y ® 3.26393 - 3.27213 ä, u ® 0.0655816 + 0.565126 ä,
v ® 2.26479 - 3.26473 ä, w ® -1.00249 - 0.23585 ä<,8x ® 0.0363031 - 0.265852 ä, y ® 0.138863 + 1.59335 ä, u ® 1.17728 - 0.216573 ä,
v ® 0.239913 + 1.03693 ä, w ® 0.105896 - 0.457693 ä<,8x ® 0.0363031 + 0.265852 ä, y ® 0.138863 - 1.59335 ä, u ® 1.17728 + 0.216573 ä,
v ® 0.239913 - 1.03693 ä, w ® 0.105896 + 0.457693 ä<,8x ® 1.87865, y ® -9.11709, u ® 2.65932, v ® -8.49214, w ® -0.058878<,8x ® 0.656674 - 0.454024 ä, y ® -0.611719 + 1.41327 ä,
u ® 0.605594 - 0.0608174 ä, v ® 0.461758 + 1.43198 ä, w ® 0.881854 + 0.178927 ä<,8x ® 0.656674 + 0.454024 ä, y ® -0.611719 - 1.41327 ä, u ® 0.605594 + 0.0608174 ä,
v ® 0.461758 - 1.43198 ä, w ® 0.881854 - 0.178927 ä<,8x ® 0.845492, y ® 0.451387, u ® 0.387633, v ® -0.437637, w ® -1.1425<,8x ® 0.740532, y ® -0.254264, u ® 0.40444, v ® 0.687565, w ® 0.727204<,8x ® 1.5995, y ® -7.59566, u ® 2.37069, v ® -6.95905, w ® 0.0593103<,8x ® 0.337506, y ® -1.09954, u ® 1.29029, v ® -0.795884, w ® 0.200219<,8x ® -0.457628, y ® 5.4572, u ® -0.520444, v ® 6.45522, w ® 1.23064<<
Im@ð@@3, 2DDD � 0, Im@ð@@4, 2DDD � 0, Im@ð@@5, 2DDD � 0D &D88x ® -2.50221, y ® 16.2682, u ® -2.1049, v ® 15.3505, w ® 0.0752342<,
8x ® -2.40824, y ® 15.8046, u ® -2.02594, v ® 14.8806, w ® 0.0336009<,8x ® 1.87865, y ® -9.11709, u ® 2.65932, v ® -8.49214, w ® -0.058878<,8x ® 0.845492, y ® 0.451387, u ® 0.387633, v ® -0.437637, w ® -1.1425<,8x ® 0.740532, y ® -0.254264, u ® 0.40444, v ® 0.687565, w ® 0.727204<,8x ® 1.5995, y ® -7.59566, u ® 2.37069, v ® -6.95905, w ® 0.0593103<,8x ® 0.337506, y ® -1.09954, u ® 1.29029, v ® -0.795884, w ® 0.200219<,8x ® -0.457628, y ® 5.4572, u ® -0.520444, v ® 6.45522, w ® 1.23064<<
To make the solution unique, one may consider the solution with minimal norm,
The problem can be transformed into a problem of minimization with constrains, namely we are looking for the solutionwith minimal norm,
AbsoluteTiming@sol = NMinimize@8Norm@8x, y, u, v, w<D, g1 == 0, g2 == 0, g3 == 0<,8x, y, u, v, w<, Method ® "DifferentialEvolution"DD
91.6093750, 90.974816,9x ® 0.187018, y ® -1.0891 ´ 10-6, u ® -0.812982, v ® 0.000124842, w ® 0.504332===
However, this result is not very encouraging,
8g1, g2, g3< �. sol@@2DD
9-7.60058 ´ 10-9, -1.47734 ´ 10-8, -4.436 ´ 10-9=
In order to overcome these problems one can extend the Newton - Raphson method using pseudoinverse of the Jacobianmatrix, which can be computed by singular value decomposition (see the next Section).
Now, we are going to introduce an extention of the Newton-Raphson method in order to avoid these difficulties. This methodwill use the pseudoinverse of the Jacobian, instead of its inverse. The computation of the pseudoinverse is based on thesingular value decomposition technique.
8- 5 Singular Value Decomposition
Every A matrix m � n, m ³ n can be decomposed as
A = U S VT
where H.LTdenotes the transposed matrix , U an m ´ n matrix, and V n ´ n matrix satifying
10 ExtendedNewton_08.nb
UT U = VT V = V VT = In
and S = < Σ1, ..., Σn > a diagonal matrix.
These Σi ’ s, Σ1 ³ Σ2 ³, ..., Σn ³ 0 are the square root of the non negative eigenvalues of AT
A
and are called as the singular values of matrix A.
As it is known from linear algebra, singular value decomposition (SVD) is a technique to compute pseudoinverse forsingular or ill-conditioned matrix of linear systems. In addition this method provides least square solution for overdeter-
mined system and minimal norm solution in case of undetermined system.
8- 6 Pseudoinverse
The pseudoinverse of a matrix A of m ´ n is a matrix A+
of n ´ m satisfying
A A+ A = A , A+ A A+ = A+, HA+ AL*= A+ A, HA A+L*
= A A+
where H.L*denotes the conjugate transpose of the matrix.
There always exists a unique A+
whic can be computed using SVD :
aL If m ³ n and A = U S VT
then
A+ = V S-1 UT
where S-1 = < 1 � Σ1, ..., 1 � Σn >
b) If m < n then compute the HATL+, pseudoinverse of AT and then
A+ = IIATM+MT
8- 7 Newton - Raphson Method with Pseudoinverse
The idea of using pseudoinverse in order to generalize of Newton-Raphson method is not new, see e.g. Quoc - Nam Tran
(1998). It means that in the iteration formula, the pseudoinverse of the Jacobian matrix will be employed,
xi+1 = xi - J+ HxiL f HxiL
In Mathematica pseudoinverse can be computed in symbolic as well as in numeric form. For example considering the firstexample in Section 8- 2- 1, the Jacobi matrix is
Fig. 8. 3 Convergence of the Extended Newton - Raphson method Hred pointsL
We have got more precise result at sligthly less required iteration steps than in case of the standard Newton-Raphson methodapplied to the determined model. This means faster convergence.
8- 9- 2 Overdetermined non-polynomial system
Now, we can solve directly the original overdetermined system of Bard, see Section 8- 2- 2,
8g1, g2, g3< �.8x ® mnsol@@1DD, y ® mnsol@@2DD, u ® mnsol@@3DD, v ® mnsol@@4DD, w ® mnsol@@5DD<
91.38778 ´ 10-17, 0., 1.38778 ´ 10-17=
However, this solution is different from the solution provided by the genetic algorithm, Extended Newton Method canprovide an effective tool to select nearly minimal norm solution from the infinite ones in relatively short computation time.Although underdetermined problems are rare in geodesy, we shall illustrate this technique in the next Section 8- 10- 3.
8- 10 Examples
8- 10- 1 Ranging LPS N-point problem in 3D
First let us consider a polynomial system. The prototype equation of the Ranging LPS N- point problem in 3D is,
This example clearly demonstrates the robustness of the method!
8- 10- 2 GPS N-point problem
Let us solve the GPS - N point problem employing the solutions of the different subsets of the Gauss- Jacobi combinatorialtechnique computed in the previous chapter. Now, we employ the distance error model, namely the general form of theequations is,
en = di - Hx1 - aiL2 + Hx2 - biL2 + Hx3 - ciL2 - x4 ;
Employing Extended Newton-Raphson method, we get the correct result independently on the initial values represented bythe different GPS-4 point subset solutions,
This example demonstrates fairly well, that employing symbolic solution via Groebner basis or Dixon Resultant for adetermined combinatorial subset system, then applying this result as an initial guess value for a numerical robust local(Extended Newton-Raphson) or a numerical global technique (Linear Homotopy) to solve overdetermined (N-point) prob-lems, can be very successful strategy for geodetical computations!
8- 10- 3 Minimum Distance Mapping
20 ExtendedNewton_08.nb
8- 10- 3 Minimum Distance Mapping
In order to relate a point P (X,Y,Z) on the Earth’s topographical surface to a point p(x,y,z) on the international referenceellipsoid, one works with a bundle of half-straight lines - so called projection lines - that depart from P and intersect theellipsoid. There is one projection line that is at minimum distance relating P to p.
P
p
a
b
Fig .8 .6 Minimum distance mapping
The distance to be minimized,
Clear@"Global‘*"D
d = HX - xL2 + HY - yL2 + HZ - zL2;
The constrain represents that the point p is an element of the ellipsoid-of revolution,
c =Ix2 + y2M
a2+
z2
b2- 1;
Instead of transforming the constrained optimization problem into an unconstrained one as is usual done, we shall solve it asan underdetermined system via Extended Newton- Raphson method. Let us introduce new variables,
Now, we have a single equation with 3 variables (Α, Β, Γ). This underdetermined problem has infinite solutions. In order toselect the proper solution we seek a solution with minimal norm, since the distance to be minimized,
d = Α2 + Β2 + Γ2
A good initial guess is (Α, Β, Γ) = {0, 0, 0}. Let us employ Extended Newton-Raphson method,
sol = NewtonExtended@8eqn<, 8Α, Β, Γ<, 80., 0., 0.<, 10^-12, 100D �� Last
826.6174, 3.14888, 36.2985<
Returning back to the original variables,
8X - Α, Y - Β, Z - Γ< �. data �. 8Α ® sol@@1DD, Β ® sol@@2DD, Γ ® sol@@3DD<
which is a quite precise solution, do compare it with the results in Chapter 18.
References
Bard Y. (1974) Nonlinear Parameter Estimation, Academic Press, New York.
Chapra S.C. and Canale R.P. (1998) Numerical Methods for Engineers, p. 159, 3rd Edition, McGraw-Hill, Boston.
Quoc - Nam Tran (1998) A Symbolic- Numerical Method for Finding a Real Solution of an Arbitrary System of NonlinearAlgebraic Equations, J. Symbolic Computation, 26, pp. 739- 760.
Ojika, T. (1987) Modified deflation algorithm for the solution of singular problems. I. A system of nonlinear algebraicequations. J.Math.Anal.Appl. 123., pp. 199 - 221