| |
| |
| Preface | |
| |
| |
| |
| Introduction. Problems to be considered | |
| |
| |
| Characteristics of 'real-world' problems | |
| |
| |
| Finite-precision arithmetic and measurement of error | |
| |
| |
| Exercises | |
| |
| |
| |
| Nonlinear Problems in One Variable | |
| |
| |
| What is not possible | |
| |
| |
| Newton's method for solving one equation in one unknown | |
| |
| |
| Convergence of sequences of real numbers | |
| |
| |
| Convergence of Newton's method | |
| |
| |
| Globally convergent methods for solving one equation in one uknown | |
| |
| |
| Methods when derivatives are unavailable | |
| |
| |
| Minimization of a function of one variable | |
| |
| |
| Exercises | |
| |
| |
| |
| Numerical Linear Algebra Background | |
| |
| |
| Vector and matrix norms and orthogonality | |
| |
| |
| Solving systems of linear equations'matrix factorizations | |
| |
| |
| Errors in solving linear systems | |
| |
| |
| Updating matrix factorizations | |
| |
| |
| Eigenvalues and positive definiteness | |
| |
| |
| Linear least squares | |
| |
| |
| Exercises | |
| |
| |
| |
| Multivariable Calculus Background | |
| |
| |
| Derivatives and multivariable models | |
| |
| |
| Multivariable finite-difference derivatives | |
| |
| |
| Necessary and sufficient conditions for unconstrained minimization | |
| |
| |
| Exercises | |
| |
| |
| |
| Newton's Method for Nonlinear Equations and Unconstrained Minimization | |
| |
| |
| Newton's method for systems of nonlinear equations | |
| |
| |
| Local convergence of Newton's method | |
| |
| |
| The Kantorovich and contractive mapping theorems | |
| |
| |
| Finite-difference derivative methods for systems of nonlinear equations | |
| |
| |
| Newton's method for unconstrained minimization | |
| |
| |
| Finite difference derivative methods for unconstrained minimization | |
| |
| |
| Exercises | |
| |
| |
| |
| Globally Convergent Modifications of Newton's Method | |
| |
| |
| The quasi-Newton framework | |
| |
| |
| Descent directions | |
| |
| |
| Line searches | |
| |
| |
| The model-trust region approach | |
| |
| |
| Global methods for systems of nonlinear equations | |
| |
| |
| Exercises | |
| |
| |
| |
| Stopping, Scaling, and Testing | |
| |
| |
| Scaling | |
| |
| |
| Stopping criteria | |
| |
| |
| Testing | |
| |
| |
| Exercises | |
| |
| |
| |
| Secant Methods for Systems of Nonlinear Equations | |
| |
| |
| Broyden's method | |
| |
| |
| Local convergence analysis of Broyden's method | |
| |
| |
| Implementation of quasi-Newton algorithms using Broyden's update | |
| |
| |
| Other secant updates for nonlinear equations | |
| |
| |
| Exercises | |
| |
| |
| |
| Secant Methods for Unconstrained Minimization | |
| |
| |
| The symmetric secant update of Powell | |
| |
| |
| Symmetric positive definite secant updates | |
| |
| |
| Local convergence of positive definite secant methods | |
| |
| |
| Implementation of quasi-Newton algorithms using the positive definite secant update | |
| |
| |
| Another convergence result for the positive definite secant method | |
| |
| |
| Other secant updates for unconstrained minimization | |
| |
| |
| Exercises | |
| |
| |
| |
| Nonlinear Least Squares | |
| |
| |
| The nonlinear least-squares problem | |
| |
| |
| Gauss-Newton-type methods | |
| |
| |
| Full Newton-type methods | |
| |
| |
| Other considerations in solving nonlinear least-squares problems | |
| |
| |
| Exercises | |
| |
| |
| |
| Methods for Problems with Special Structure | |
| |
| |
| The sparse finite-difference Newton method | |
| |
| |
| Sparse secant methods | |
| |
| |
| Deriving least-change secant updates | |
| |
| |
| Analyzing least-change secant methods | |
| |
| |
| Exercises | |
| |
| |
| |
| A Modular System of Algorithms for Unconstrained Minimization and Nonlinear Equations | |
| |
| |
| |
| |
| Test Problems | |
| |
| |
| |
| References | |
| |
| |
| Author Index | |
| |
| |
| Subject Index | |