This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1/18/12 Solve system of nonlinear equations - MATLAB
for x, where x is a vector and F(x) is a function that returns a vector value.
Syntax
x = f s o l v e ( f u n , x 0 )x = f s o l v e ( f u n , x 0 , o p t i o n s )x = f s o l v e ( p r o b l e m )[ x , f v a l ] = f s o l v e ( f u n , x 0 )[ x , f v a l , e x i t f l a g ] = f s o l v e ( . . . )[ x , f v a l , e x i t f l a g , o u t p u t ] = f s o l v e ( . . . )[ x , f v a l , e x i t f l a g , o u t p u t , j a c o b i a n ] = f s o l v e ( . . . )
Description
f s o l v e finds a root (zero) of a system of nonlinear equations.
Note Passing Extra Parameters explains how to pass extra parameters to the system of equations, if necessary.
x = f s o l v e ( f u n , x 0 ) starts at x 0 and tries to solve the equations described in f u n .
x = f s o l v e ( f u n , x 0 , o p t i o n s ) solves the equations with the optimization options specified in the structure o p t i o n s . Use o p t i m s e t to set these options.
x = f s o l v e ( p r o b l e m ) solves p r o b l e m , where p r o b l e m is a structure described in Input Arguments.
Create the structure p r o b l e m by exporting a problem from Optimization Tool, as described in Exporting Your Work.
[ x , f v a l ] = f s o l v e ( f u n , x 0 ) returns the value of the objective function f u n at the solution x .
[ x , f v a l , e x i t f l a g ] = f s o l v e ( . . . ) returns a value e x i t f l a g that describes the exit condition.
[ x , f v a l , e x i t f l a g , o u t p u t ] = f s o l v e ( . . . ) returns a structure o u t p u t that contains information about the optimization.
[ x , f v a l , e x i t f l a g , o u t p u t , j a c o b i a n ] = f s o l v e ( . . . ) returns the Jacobian of f u n at the solution x .
Input Arguments
Function Arguments contains general descriptions of arguments passed into f s o l v e . This section provides function-specific details for f u n and p r o b l e m :
f u n The nonlinear system of equations to solve. f u n is a function that accepts a vector x and returns a vector F , the nonlinear
equations evaluated at x . The function f u n can be specified as a function handle for a file
x = f s o l v e ( @ m y f u n , x 0 )
where m y f u n is a MATLAB function such as
f u n c t i o n F = m y f u n ( x )F = . . . % C o m p u t e f u n c t i o n v a l u e s a t x
f u n can also be a function handle for an anonymous function.
x = f s o l v e ( @ ( x ) s i n ( x . * x ) , x 0 ) ;
If the user-defined values for x and F are matrices, they are converted to a vector using linear indexing.
If the Jacobian can also be computed and the Jacobian option is ' o n ' , set by
o p t i o n s = o p t i m s e t ( ' J a c o b i a n ' , ' o n ' )
the function f u n must return, in a second output argument, the Jacobian value J , a matrix, at x .
If f u n returns a vector (matrix) of m components and x has length n , where n is the length of x 0 , the Jacobian J is an m -by-nmatrix where J ( i , j ) is the partial derivative of F ( i ) with respect to x ( j ) . (The Jacobian J is the transpose of the gradient of F .)
p r o b l e m o b j e c t i v e Objective function
x 0 Initial point for x
s o l v e r ' f s o l v e '
o p t i o n s Options structure created with o p t i m s e t
Output Arguments
Function Arguments contains general descriptions of arguments returned by f s o l v e . For more information on the output headings for f s o l v e , see
Function-Specific Headings.
This section provides function-specific details for e x i t f l a g and o u t p u t :
1/18/12 Solve system of nonlinear equations - MATLAB
e x i t f l a g Integer identifying the reason the algorithm terminated. The following lists the values of e x i t f l a g and the corresponding
reasons the algorithm terminated.
1 Function converged to a solution x .
2 Change in x was smaller than the specified tolerance.
3 Change in the residual was smaller than the specified tolerance.
4 Magnitude of search direction was smaller than the specified tolerance.
0 Number of iterations exceeded o p t i o n s . M a x I t e r or number of function evaluations
exceeded o p t i o n s . F u n E v a l s .
- 1 Output function terminated the algorithm.
- 2 Algorithm appears to be converging to a point that is not a root.
- 3 Trust region radius became too small (t r u s t - r e g i o n - d o g l e g algorithm) or
regularization parameter became too large (l e v e n b e r g - m a r q u a r d t algorithm).
- 4 Line search cannot sufficiently decrease the residual along the current searchdirection.
o u t p u t Structure containing information about the optimization. The fields of the structure are
i t e r a t i o n s Number of iterations taken
f u n c C o u n t Number of function evaluations
a l g o r i t h m Optimization algorithm used
c g i t e r a t i o n s Total number of PCG iterations (large-scale algorithm only)
s t e p s i z e Final displacement in x (Levenberg-Marquardt algorithm)
f i r s t o r d e r o p t Measure of first-order optimality (dogleg or large-scale algorithm, [ ] for others)
m e s s a g e Exit message
Options
Optimization options used by f s o l v e . Some options apply to all algorithms, some are only relevant when using the trust-region-reflective algorithm, and
others are only relevant when using the other algorithms. You can use o p t i m s e t to set or change the values of these fields in the options structure,
o p t i o n s . See Optimization Options Reference for detailed information.
All Algorithms
All algorithms use the following options:
A l g o r i t h m Choose between ' t r u s t - r e g i o n - d o g l e g ' (default), ' t r u s t - r e g i o n - r e f l e c t i v e ' , and ' l e v e n b e r g -m a r q u a r d t ' . Set the initial Levenberg-Marquardt parameter NJ by setting A l g o r i t h m to a cell array such as
{ ' l e v e n b e r g - m a r q u a r d t ' , . 0 0 5 } . The default NJ = 0 . 0 1 .
The A l g o r i t h m option specifies a preference for which algorithm to use. It is only a preference because for the
trust-region-reflective algorithm, the nonlinear system of equations cannot be underdetermined; that is, the
number of equations (the number of elements of F returned by f u n ) must be at least as many as the length of
x . Similarly, for the trust-region-dogleg algorithm, the number of equations must be the same as the length of
x . f s o l v e uses the Levenberg-Marquardt algorithm when the selected algorithm is unavailable. For more
information on choosing the algorithm, see Choosing the Algorithm.
D e r i v a t i v e C h e c k Compare user-supplied derivatives (gradients of objective or constraints) to finite-differencing derivatives. The
choices are ' o n ' or the default ' o f f ' .
D i a g n o s t i c s Display diagnostic information about the function to be minimized or solved. The choices are ' o n ' or the
default ' o f f ' .
D i f f M a x C h a n g e Maximum change in variables for finite-difference gradients (a positive scalar). The default is I n f .
D i f f M i n C h a n g e Minimum change in variables for finite-difference gradients (a positive scalar). The default is 0 .
D i s p l a y Level of display:
' o f f ' displays no output.
' i t e r ' displays output at each iteration.
1/18/12 Solve system of nonlinear equations - MATLAB
' n o t i f y ' displays output only if the function does not converge.
' f i n a l ' (default) displays just the final output.
F i n D i f f R e l S t e p Scalar or vector step size factor. When you set F i n D i f f R e l S t e p to a vector v , forward finite differences d e l t aare
d e l t a = v . * s i g n ( x ) . * m a x ( a b s ( x ) , T y p i c a l X ) ;
and central finite differences are
d e l t a = v . * m a x ( a b s ( x ) , T y p i c a l X ) ;
Scalar F i n D i f f R e l S t e p expands to a vector. The default is s q r t ( e p s ) for forward finite differences, and
e p s ^ ( 1 / 3 ) for central finite differences.
F i n D i f f T y p e Finite differences, used to estimate gradients, are either ' f o r w a r d ' (default), or ' c e n t r a l ' (centered).
' c e n t r a l ' takes twice as many function evaluations, but should be more accurate.
The algorithm is careful to obey bounds when estimating both types of finite differences. So, for example, itcould take a backward, rather than a forward, difference to avoid evaluating at a point outside bounds.
F u n V a l C h e c k Check whether objective function values are valid. ' o n ' displays an error when the objective function returns a
value that is c o m p l e x , I n f , or N a N . The default, ' o f f ' , displays no error.
J a c o b i a n If ' o n ' , f s o l v e uses a user-defined Jacobian (defined in f u n ), or Jacobian information (when using
J a c o b M u l t ), for the objective function. If ' o f f ' (default), f s o l v e approximates the Jacobian using finite
differences.
M a x F u n E v a l s Maximum number of function evaluations allowed, a positive integer. The default is 1 0 0 * n u m b e r O f V a r i a b l e s .
M a x I t e r Maximum number of iterations allowed, a positive integer. The default is 4 0 0 .
O u t p u t F c n Specify one or more user-defined functions that an optimization function calls at each iteration, either as a
function handle or as a cell array of function handles. The default is none ([ ] ). See Output Function.
P l o t F c n s Plots various measures of progress while the algorithm executes. Select from predefined plots or write your
own. Pass a function handle or a cell array of function handles. The default is none ([ ] ):
@ o p t i m p l o t x plots the current point.
@ o p t i m p l o t f u n c c o u n t plots the function count.
@ o p t i m p l o t f v a l plots the function value.
@ o p t i m p l o t r e s n o r m plots the norm of the residuals.
@ o p t i m p l o t s t e p s i z e plots the step size.
@ o p t i m p l o t f i r s t o r d e r o p t plots the first-order optimality measure.
For information on writing a custom plot function, see Plot Functions.
T o l F u n Termination tolerance on the function value, a positive scalar. The default is 1 e - 6 .
T o l X Termination tolerance on x , a positive scalar. The default is 1 e - 6 .
T y p i c a l X Typical x values. The number of elements in T y p i c a l X is equal to the number of elements in x 0 , the starting
point. The default value is o n e s ( n u m b e r o f v a r i a b l e s , 1 ) . f s o l v e uses T y p i c a l X for scaling finite differences
for gradient estimation.
The t r u s t - r e g i o n - d o g l e g algorithm uses T y p i c a l X as the diagonal terms of a scaling matrix.
Trust-Region-Reflective Algorithm Only
The trust-region-reflective algorithm uses the following options:
J a c o b M u l t Function handle for Jacobian multiply function. For large-scale structured problems, this function
computes the Jacobian matrix product J * Y , J ' * Y , or J ' * ( J * Y ) without actually forming J . The function is
of the form
W = j m f u n ( J i n f o , Y , f l a g )
where J i n f o contains a matrix used to compute J * Y (or J ' * Y , or J ' * ( J * Y ) ). The first argument J i n f omust be the same as the second argument returned by the objective function f u n , for example, in
[ F , J i n f o ] = f u n ( x )
Y is a matrix that has the same number of rows as there are dimensions in the problem. f l a g determines
which product to compute:
If f l a g = = 0 , W = J ' * ( J * Y ) .
If f l a g > 0 , W = J * Y .
If f l a g < 0 , W = J ' * Y .
In each case, J is not formed explicitly. f s o l v e uses J i n f o to compute the preconditioner. See Passing
Extra Parameters for information on how to supply values for any additional parameters j m f u n needs.
Note ' J a c o b i a n ' must be set to ' o n ' for f s o l v e to pass J i n f o from f u n to j m f u n .
1/18/12 Solve system of nonlinear equations - MATLAB
See Example: Nonlinear Minimization with a Dense but Structured Hessian and Equality Constraints for asimilar example.
J a c o b P a t t e r n Sparsity pattern of the Jacobian for finite differencing. If it is not convenient to compute the Jacobian
matrix J in f u n , l s q n o n l i n can approximate J via sparse finite differences, provided you supply the
structure of J (i.e., locations of the nonzeros) as the value for J a c o b P a t t e r n . In the worst case, if the
structure is unknown, you can set J a c o b P a t t e r n to be a dense matrix and a full finite-difference
approximation is computed in each iteration (this is the default if J a c o b P a t t e r n is not set). This can be
very expensive for large problems, so it is usually worth the effort to determine the sparsity structure.
M a x P C G I t e r Maximum number of PCG (preconditioned conjugate gradient) iterations, a positive scalar. The default is
m a x ( 1 , f l o o r ( n u m b e r O f V a r i a b l e s / 2 ) ) . For more information, see Algorithms.
P r e c o n d B a n d W i d t h Upper bandwidth of preconditioner for PCG, a nonnegative integer. The default P r e c o n d B a n d W i d t h is
I n f , which means a direct factorization (Cholesky) is used rather than the conjugate gradients (CG). The
direct factorization is computationally more expensive than CG, but produces a better quality step
towards the solution. Set P r e c o n d B a n d W i d t h to 0 for diagonal preconditioning (upper bandwidth of 0).
For some problems, an intermediate bandwidth reduces the number of PCG iterations.
T o l P C G Termination tolerance on the PCG iteration, a positive scalar. The default is 0 . 1 .
Levenberg-Marquardt Algorithm Only
The Levenberg-Marquardt algorithm uses the following option:
S c a l e P r o b l e m ' J a c o b i a n ' can sometimes improve the convergence of a poorly scaled problem. The default is
' n o n e ' .
Examples
Example 1
This example solves the system of two equations and two unknowns:
Rewrite the equations in the form F(x) = 0:
Start your search for a solution at x 0 = [ - 5 - 5 ] .
First, write a file that computes F , the values of the equations at x .
f u n c t i o n F = m y f u n ( x )F = [ 2 * x ( 1 ) - x ( 2 ) - e x p ( - x ( 1 ) ) ; - x ( 1 ) + 2 * x ( 2 ) - e x p ( - x ( 2 ) ) ] ;
Save this function file as m y f u n . m somewhere on your MATLAB path. Next, set up the initial point and options and call f s o l v e :
x 0 = [ - 5 ; - 5 ] ; % M a k e a s t a r t i n g g u e s s a t t h e s o l u t i o no p t i o n s = o p t i m s e t ( ' D i s p l a y ' , ' i t e r ' ) ; % O p t i o n t o d i s p l a y o u t p u t[ x , f v a l ] = f s o l v e ( @ m y f u n , x 0 , o p t i o n s ) % C a l l s o l v e r
After several iterations, f s o l v e finds an answer:
N o r m o f F i r s t - o r d e r T r u s t - r e g i o nI t e r a t i o n F u n c - c o u n t f ( x ) s t e p o p t i m a l i t y r a d i u s 0 3 2 3 5 3 5 . 6 2 . 2 9 e + 0 0 4 1 1 6 6 0 0 1 . 7 2 1 5 . 7 5 e + 0 0 3 1 2 9 1 5 7 3 . 5 1 1 1 . 4 7 e + 0 0 3 1 3 1 2 4 2 7 . 2 2 6 1 3 8 8 1 4 1 5 1 1 9 . 7 6 3 1 1 0 7 1 5 1 8 3 3 . 5 2 0 6 1 3 0 . 8 1 6 2 1 8 . 3 5 2 0 8 1 9 . 0 5 1 7 2 4 1 . 2 1 3 9 4 1 2 . 2 6 1 8 2 7 0 . 0 1 6 3 2 9 0 . 7 5 9 5 1 1 0 . 2 0 6 2 . 5 9 3 0 3 . 5 1 5 7 5 e - 0 0 6 0 . 1 1 1 9 2 7 0 . 0 0 2 9 4 2 . 5 1 0 3 3 1 . 6 4 7 6 3 e - 0 1 3 0 . 0 0 1 6 9 1 3 2 6 . 3 6 e - 0 0 7 2 . 5
E q u a t i o n s o l v e d .
f s o l v e c o m p l e t e d b e c a u s e t h e v e c t o r o f f u n c t i o n v a l u e s i s n e a r z e r oa s m e a s u r e d b y t h e d e f a u l t v a l u e o f t h e f u n c t i o n t o l e r a n c e , a n dt h e p r o b l e m a p p e a r s r e g u l a r a s m e a s u r e d b y t h e g r a d i e n t .
1/18/12 Solve system of nonlinear equations - MATLAB
f v a l = 1 . 0 e - 0 0 6 * - 0 . 4 0 5 9 - 0 . 4 0 5 9
Example 2
Find a matrix x that satisfies the equation
starting at the point x = [ 1 , 1 ; 1 , 1 ] .
First, write a file that computes the equations to be solved.
f u n c t i o n F = m y f u n ( x )F = x * x * x - [ 1 , 2 ; 3 , 4 ] ;
Save this function file as m y f u n . m somewhere on your MATLAB path. Next, set up an initial point and options and call f s o l v e :
x 0 = o n e s ( 2 , 2 ) ; % M a k e a s t a r t i n g g u e s s a t t h e s o l u t i o no p t i o n s = o p t i m s e t ( ' D i s p l a y ' , ' o f f ' ) ; % T u r n o f f d i s p l a y[ x , F v a l , e x i t f l a g ] = f s o l v e ( @ m y f u n , x 0 , o p t i o n s )
F v a l = 1 . 0 e - 0 0 9 * - 0 . 1 6 2 1 0 . 0 7 8 0 0 . 1 1 6 4 - 0 . 0 4 6 7
e x i t f l a g = 1
and the residual is close to zero.
s u m ( s u m ( F v a l . * F v a l ) )a n s = 4 . 8 0 8 1 e - 0 2 0
Notes
If the system of equations is linear, use \ (matrix left division) for better speed and accuracy. For example, to find the solution to the following linear
The Levenberg-Marquardt and trust-region-reflective methods are based on the nonlinear least-squares algorithms also used in l s q n o n l i n . Use one of
these methods if the system may not have a zero. The algorithm still returns a point where the residual is small. However, if the Jacobian of the system issingular, the algorithm might converge to a point that is not a solution of the system of equations (see Limitations and Diagnostics following).
By default f s o l v e chooses the trust-region dogleg algorithm. The algorithm is a variant of the Powell dogleg method described in [8]. It is similar in
nature to the algorithm implemented in [7]. It is described in the User's Guide in Trust-Region Dogleg Method.
The trust-region-reflective algorithm is a subspace trust-region method and is based on the interior-reflective Newton method described in [1] and [2].Each iteration involves the approximate solution of a large linear system using the method of preconditioned conjugate gradients (PCG). See Trust-Region Reflective fsolve Algorithm.
The Levenberg-Marquardt method is described in [4], [5], and [6]. It is described in the User's Guide in Levenberg-Marquardt Method.
Diagnostics
All Algorithms
f s o l v e may converge to a nonzero point and give this message:
1/18/12 Solve system of nonlinear equations - MATLAB
Learn how to use optimization tosolve systems of equations, fitmodels to data, or optimize systemperformance.
Get free kit
Trials Available
Try the latest version ofoptimization products.
Get trial software
O p t i m i z e r i s s t u c k a t a m i n i m u m t h a t i s n o t a r o o tT r y a g a i n w i t h a n e w s t a r t i n g g u e s s
In this case, run f s o l v e again with other starting values.
Trust-Region-Dogleg Algorithm
For the trust-region dogleg method, f s o l v e stops if the step size becomes too small and it can make no more progress. f s o l v e gives this message:
T h e o p t i m i z a t i o n a l g o r i t h m c a n m a k e n o f u r t h e r p r o g r e s s : T r u s t r e g i o n r a d i u s l e s s t h a n 1 0 * e p s
In this case, run f s o l v e again with other starting values.
Limitations
The function to be solved must be continuous. When successful, f s o l v e only gives one root. f s o l v e may converge to a nonzero point, in which case, try
other starting values.
f s o l v e only handles real variables. When x has complex variables, the variables must be split into real and imaginary parts.
Trust-Region-Reflective Algorithm
The preconditioner computation used in the preconditioned conjugate gradient part of the trust-region-reflective algorithm forms J7J (where J is theJacobian matrix) before computing the preconditioner; therefore, a row of J with many nonzeros, which results in a nearly dense product J7J, might lead toa costly solution process for large problems.
Large-Scale Problem Coverage and Requirements
For Large Problems
Provide sparsity structure of the Jacobian or compute the Jacobian in f u n .
The Jacobian should be sparse.
Number of Equations
The default trust-region dogleg method can only be used when the system of equations is square, i.e., the number of equations equals the number ofunknowns. For the Levenberg-Marquardt method, the system of equations need not be square.
References
[1] Coleman, T.F. and Y. Li, "An Interior, Trust Region Approach for Nonlinear Minimization Subject to Bounds," SIAM Journal on Optimization, Vol. 6, pp.418-445, 1996.
[2] Coleman, T.F. and Y. Li, "On the Convergence of Reflective Newton Methods for Large-Scale Nonlinear Minimization Subject to Bounds," MathematicalProgramming, Vol. 67, Number 2, pp. 189-224, 1994.
[3] Dennis, J. E. Jr., "Nonlinear Least-Squares," State of the Art in Numerical Analysis, ed. D. Jacobs, Academic Press, pp. 269-312.
[4] Levenberg, K., "A Method for the Solution of Certain Problems in Least-Squares," Quarterly Applied Mathematics 2, pp. 164-168, 1944.
[5] Marquardt, D., "An Algorithm for Least-squares Estimation of Nonlinear Parameters," SIAM Journal Applied Mathematics, Vol. 11, pp. 431-441, 1963.
[6] Moré, J. J., "The Levenberg-Marquardt Algorithm: Implementation and Theory," Numerical Analysis, ed. G. A. Watson, Lecture Notes in Mathematics630, Springer Verlag, pp. 105-116, 1977.
[7] Moré, J. J., B. S. Garbow, and K. E. Hillstrom, User Guide for MINPACK 1, Argonne National Laboratory, Rept. ANL-80-74, 1980.
[8] Powell, M. J. D., "A Fortran Subroutine for Solving Systems of Nonlinear Algebraic Equations," Numerical Methods for Nonlinear Algebraic Equations, P.Rabinowitz, ed., Ch.7, 1970.
See Also
l s q c u r v e f i t | l s q n o n l i n | o p t i m s e t | o p t i m t o o l
How To
@ ( f u n c t i o n _ h a n d l e )\Anonymous FunctionsEquation Solving AlgorithmsEquation Solving Examples