| |
| |
Preface | |
| |
| |
| |
Mathematical Review | |
| |
| |
| |
Methods of Proof and Some Notation | |
| |
| |
| |
Methods of Proof | |
| |
| |
| |
Notation | |
| |
| |
Exercises | |
| |
| |
| |
Vector Spaces and Matrices | |
| |
| |
| |
Vector and Matrix | |
| |
| |
| |
Rank of a Matrix | |
| |
| |
| |
Linear Equations | |
| |
| |
| |
Inner Products and Norms | |
| |
| |
Exercises | |
| |
| |
| |
Transformations | |
| |
| |
| |
Linear Transformations | |
| |
| |
| |
Eigenvalues and Eigenvectors | |
| |
| |
| |
Orthogonal Projections | |
| |
| |
| |
Quadratic Forms | |
| |
| |
| |
Matrix Norms | |
| |
| |
Exercises | |
| |
| |
| |
Concepts from Geometry | |
| |
| |
| |
Line Segments | |
| |
| |
| |
Hyperplanes and Linear Varieties | |
| |
| |
| |
Convex Sets | |
| |
| |
| |
Neighborhoods | |
| |
| |
| |
Polytopes and Polyhedra | |
| |
| |
Exercises | |
| |
| |
| |
Elements of Calculus | |
| |
| |
| |
Sequences and Limits | |
| |
| |
| |
Differentiability | |
| |
| |
| |
The Derivative Matrix | |
| |
| |
| |
Differentiation Rules | |
| |
| |
| |
Level Sets and Gradients | |
| |
| |
| |
Taylor Series | |
| |
| |
Exercises | |
| |
| |
| |
Unconstrained Optimization | |
| |
| |
| |
Basics of Set-Constrained and Unconstrained Optimization | |
| |
| |
| |
Introduction | |
| |
| |
| |
Conditions for Local Minimizers | |
| |
| |
Exercises | |
| |
| |
| |
One-Dimensional Search Methods | |
| |
| |
| |
Golden Section Search | |
| |
| |
| |
Fibonacci Search | |
| |
| |
| |
Newton's Method | |
| |
| |
| |
Secant Method | |
| |
| |
| |
Remarks on Line Search Methods | |
| |
| |
Exercises | |
| |
| |
| |
Gradient Methods | |
| |
| |
| |
Introduction | |
| |
| |
| |
The Method of Steepest Descent | |
| |
| |
| |
Analysis of Gradient Methods | |
| |
| |
Exercises | |
| |
| |
| |
Newton's Method | |
| |
| |
| |
Introduction | |
| |
| |
| |
Analysis of Newton's Method | |
| |
| |
| |
Levenberg-Marquardt Modification | |
| |
| |
| |
Newton's Method for Nonlinear Least Squares | |
| |
| |
Exercises | |
| |
| |
| |
Conjugate Direction Methods | |
| |
| |
| |
Introduction | |
| |
| |
| |
The Conjugate Direction Algorithm | |
| |
| |
| |
The Conjugate Gradient Algorithm | |
| |
| |
| |
The Conjugate Gradient Algorithm for Nonquadratic | |
| |
| |
Problems | |
| |
| |
Exercises | |
| |
| |
| |
Quasi-Newton Methods | |
| |
| |
| |
Introduction | |
| |
| |
| |
Approximating the Inverse Hessian | |
| |
| |
| |
The Rank One Correction Formula | |
| |
| |
| |
The DFP Algorithm | |
| |
| |
| |
The BFGS Algorithm | |
| |
| |
Exercises | |
| |
| |
| |
Solving Linear Equations | |
| |
| |
| |
Least-Squares Analysis | |
| |
| |
| |
The Recursive Least-Squares Algorithm | |
| |
| |
| |
Solution to a Linear Equation with Minimum Norm | |
| |
| |
| |
Kaczmarz's Algorithm | |
| |
| |
| |
Solving Linear Equations in General | |
| |
| |
Exercises | |
| |
| |
| |
Unconstrained Optimization and Neural Networks | |
| |
| |
| |
Introduction | |
| |
| |
| |
Single-Neuron Training | |
| |
| |
| |
The Backpropagation Algorithm | |
| |
| |
Exercises | |
| |
| |
| |
Global Search Algorithms | |
| |
| |
| |
Introduction | |
| |
| |
| |
The Nelder-Mead Simplex Algorithm | |
| |
| |
| |
Simulated Annealing | |
| |
| |
| |
Particle Swarm Optimization | |
| |
| |
| |
Genetic Algorithms | |
| |
| |
Exercises | |
| |
| |
| |
Linear Programming | |
| |
| |
| |
Introduction to Linear Programming | |
| |
| |
| |
Brief History of Linear Programming | |
| |
| |
| |
Simple Examples of Linear Programs | |
| |
| |
| |
Two-Dimensional Linear Programs | |
| |
| |
| |
Convex Polyhedra and Linear Programming | |
| |
| |
| |
Standard Form Linear Programs | |
| |
| |
| |
Basic Solutions | |
| |
| |
| |
Properties of Basic Solutions | |
| |
| |
| |
Geometric View of Linear Programs | |
| |
| |
Exercises | |
| |
| |
| |
Simplex Method | |
| |
| |
| |
Solving Linear Equations Using Row Operations | |
| |
| |
| |
The Canonical Augmented Matrix | |
| |
| |
| |
Updating the Augmented Matrix | |
| |
| |
| |
The Simplex Algorithm | |
| |
| |
| |
Matrix Form of the Simplex Method | |
| |
| |
| |
Two-Phase Simplex Method | |
| |
| |
| |
Revised Simplex Method | |
| |
| |
Exercises | |
| |
| |
| |
Duality | |
| |
| |
| |
Dual Linear Programs | |
| |
| |
| |
Properties of Dual Problems | |
| |
| |
Exercises | |
| |
| |
| |
Nonsimplex Methods | |
| |
| |
| |
Introduction | |
| |
| |
| |
Khachiyan's Method | |
| |
| |
| |
Affine Scaling Method | |
| |
| |
| |
Karmarkar's Method | |
| |
| |
Exercises | |
| |
| |
| |
Nonlinear Constrained Optimization | |
| |
| |
| |
Problems with Equality Constraints | |
| |
| |
| |
Introduction | |
| |
| |
| |
Problem Formulation | |
| |
| |
| |
Tangent and Normal Spaces | |
| |
| |
| |
Lagrange Condition | |
| |
| |
| |
Second-Order Conditions | |
| |
| |
| |
Minimizing Quadratics Subject to Linear Constraints | |
| |
| |
Exercises | |
| |
| |
| |
Problems with Inequality Constraints | |
| |
| |
| |
Karush-Kuhn-Tucker Condition | |
| |
| |
| |
Second-Order Conditions | |
| |
| |
Exercises | |
| |
| |
| |
Convex Optimization Problems | |
| |
| |
| |
Introduction | |
| |
| |
| |
Convex Functions | |
| |
| |
| |
Convex Optimization Problems | |
| |
| |
| |
Semidefinite Programming | |
| |
| |
Exercises | |
| |
| |
| |
Algorithms for Constrained Optimization | |
| |
| |
| |
Introduction | |
| |
| |
| |
Projections | |
| |
| |
| |
Projected Gradient Methods with Linear Constraints | |
| |
| |
| |
Lagrangian Algorithms | |
| |
| |
| |
Penalty Methods | |
| |
| |
Exercises | |
| |
| |
| |
Multiobjective Optimization | |
| |
| |
| |
Introduction | |
| |
| |
| |
Pareto Solutions | |
| |
| |
| |
Computing the Pareto Front | |
| |
| |
| |
From Multiobjective to Single-Objective Optimization | |
| |
| |
| |
Uncertain Linear Programming Problems | |
| |
| |
Exercises | |
| |
| |
References | |
| |
| |
Index | |