| |
| |
Preface | |
| |
| |
| |
Introduction. Problems to be considered | |
| |
| |
Characteristics of 'real-world' problems | |
| |
| |
Finite-precision arithmetic and measurement of error | |
| |
| |
Exercises | |
| |
| |
| |
Nonlinear Problems in One Variable | |
| |
| |
What is not possible | |
| |
| |
Newton's method for solving one equation in one unknown | |
| |
| |
Convergence of sequences of real numbers | |
| |
| |
Convergence of Newton's method | |
| |
| |
Globally convergent methods for solving one equation in one uknown | |
| |
| |
Methods when derivatives are unavailable | |
| |
| |
Minimization of a function of one variable | |
| |
| |
Exercises | |
| |
| |
| |
Numerical Linear Algebra Background | |
| |
| |
Vector and matrix norms and orthogonality | |
| |
| |
Solving systems of linear equations'matrix factorizations | |
| |
| |
Errors in solving linear systems | |
| |
| |
Updating matrix factorizations | |
| |
| |
Eigenvalues and positive definiteness | |
| |
| |
Linear least squares | |
| |
| |
Exercises | |
| |
| |
| |
Multivariable Calculus Background | |
| |
| |
Derivatives and multivariable models | |
| |
| |
Multivariable finite-difference derivatives | |
| |
| |
Necessary and sufficient conditions for unconstrained minimization | |
| |
| |
Exercises | |
| |
| |
| |
Newton's Method for Nonlinear Equations and Unconstrained Minimization | |
| |
| |
Newton's method for systems of nonlinear equations | |
| |
| |
Local convergence of Newton's method | |
| |
| |
The Kantorovich and contractive mapping theorems | |
| |
| |
Finite-difference derivative methods for systems of nonlinear equations | |
| |
| |
Newton's method for unconstrained minimization | |
| |
| |
Finite difference derivative methods for unconstrained minimization | |
| |
| |
Exercises | |
| |
| |
| |
Globally Convergent Modifications of Newton's Method | |
| |
| |
The quasi-Newton framework | |
| |
| |
Descent directions | |
| |
| |
Line searches | |
| |
| |
The model-trust region approach | |
| |
| |
Global methods for systems of nonlinear equations | |
| |
| |
Exercises | |
| |
| |
| |
Stopping, Scaling, and Testing | |
| |
| |
Scaling | |
| |
| |
Stopping criteria | |
| |
| |
Testing | |
| |
| |
Exercises | |
| |
| |
| |
Secant Methods for Systems of Nonlinear Equations | |
| |
| |
Broyden's method | |
| |
| |
Local convergence analysis of Broyden's method | |
| |
| |
Implementation of quasi-Newton algorithms using Broyden's update | |
| |
| |
Other secant updates for nonlinear equations | |
| |
| |
Exercises | |
| |
| |
| |
Secant Methods for Unconstrained Minimization | |
| |
| |
The symmetric secant update of Powell | |
| |
| |
Symmetric positive definite secant updates | |
| |
| |
Local convergence of positive definite secant methods | |
| |
| |
Implementation of quasi-Newton algorithms using the positive definite secant update | |
| |
| |
Another convergence result for the positive definite secant method | |
| |
| |
Other secant updates for unconstrained minimization | |
| |
| |
Exercises | |
| |
| |
| |
Nonlinear Least Squares | |
| |
| |
The nonlinear least-squares problem | |
| |
| |
Gauss-Newton-type methods | |
| |
| |
Full Newton-type methods | |
| |
| |
Other considerations in solving nonlinear least-squares problems | |
| |
| |
Exercises | |
| |
| |
| |
Methods for Problems with Special Structure | |
| |
| |
The sparse finite-difference Newton method | |
| |
| |
Sparse secant methods | |
| |
| |
Deriving least-change secant updates | |
| |
| |
Analyzing least-change secant methods | |
| |
| |
Exercises | |
| |
| |
| |
A Modular System of Algorithms for Unconstrained Minimization and Nonlinear Equations | |
| |
| |
| |
| |
Test Problems | |
| |
| |
| |
References | |
| |
| |
Author Index | |
| |
| |
Subject Index | |