| |

| |

Preface | |

| |

| |

Acknowledgments | |

| |

| |

Author | |

| |

| |

| |

Background and Introduction | |

| |

| |

| |

Background | |

| |

| |

| |

Differential Equations | |

| |

| |

| |

Linear First-Order ODEs | |

| |

| |

| |

Second-Order ODEs with Constant Coefficients | |

| |

| |

| |

Matrix Analysis | |

| |

| |

| |

Matrix Operations | |

| |

| |

| |

Special Matrices | |

| |

| |

| |

Determinant of a Matrix | |

| |

| |

| |

Inverse of a Matrix | |

| |

| |

| |

Matrix Eigenvalue Problem | |

| |

| |

| |

Solving the Eigenvalue Problem | |

| |

| |

| |

Similarity Transformation | |

| |

| |

| |

Introduction to Numerical Methods | |

| |

| |

| |

Errors and Approximations | |

| |

| |

| |

Computational Errors | |

| |

| |

| |

Binary and Hexadecimal Numbers | |

| |

| |

| |

Floating Point and Rounding Errors | |

| |

| |

| |

Absolute and Relative Errors | |

| |

| |

| |

Iterative Methods | |

| |

| |

| |

A Fundamental Iterative Method | |

| |

| |

| |

Rate of Convergence of an Iterative Method | |

| |

| |

Problem Set | |

| |

| |

| |

Introduction to MATLABï¿½ | |

| |

| |

| |

MATLABï¿½ Built-in Functions | |

| |

| |

| |

Rounding Commands | |

| |

| |

| |

Relational Operators | |

| |

| |

| |

Format Options | |

| |

| |

| |

Vectors and Matrices | |

| |

| |

| |

Linspace | |

| |

| |

| |

Matrices | |

| |

| |

| |

Determinant, Transpose, and Inverse | |

| |

| |

| |

Slash Operators | |

| |

| |

| |

Element-by-Element Operations | |

| |

| |

| |

User-Defined Functions and Script Files | |

| |

| |

| |

Setting Default Values for Input Variables | |

| |

| |

| |

Creating Script Files | |

| |

| |

| |

Anonymous Functions | |

| |

| |

| |

Inline | |

| |

| |

| |

Program Flow Control | |

| |

| |

| |

for Loop | |

| |

| |

| |

if Command | |

| |

| |

| |

while Loop | |

| |

| |

| |

Displaying Formatted Data | |

| |

| |

| |

Symbolic Toolbox | |

| |

| |

| |

Differentiation | |

| |

| |

| |

Integration | |

| |

| |

| |

Differential Equations | |

| |

| |

| |

Plotting | |

| |

| |

| |

Subplot | |

| |

| |

| |

Plotting Analytical Expressions | |

| |

| |

| |

Multiple Plots | |

| |

| |

Problem Set | |

| |

| |

| |

Solution of Equations of a Single Variable | |

| |

| |

| |

Numerical Solution of Equations | |

| |

| |

| |

Bisection Method | |

| |

| |

| |

MATLABï¿½ Built-in Function fzero | |

| |

| |

| |

Regula Falsi Method (Method of False Position) | |

| |

| |

| |

Modified Regula Falsi Method | |

| |

| |

| |

Fixed-Point Method | |

| |

| |

| |

Selection of a Suitable Iteration Function | |

| |

| |

| |

A Note on Convergence | |

| |

| |

| |

Rate of Convergence of the Fixed-Point Iteration | |

| |

| |

| |

Newton's Method (Newton-Raphson Method) | |

| |

| |

| |

Rate of Convergence of Newton's Method | |

| |

| |

| |

A Few Notes on Newton's Method | |

| |

| |

| |

Modified Newton's Method for Roots with Multiplicity 2 or Higher | |

| |

| |

| |

Secant Method | |

| |

| |

| |

Rate of Convergence of Secant Method | |

| |

| |

| |

A Few Notes on Secant Method | |

| |

| |

| |

Equations with Several Roots | |

| |

| |

| |

Finding Zeros to the Right of a Specified Point | |

| |

| |

| |

Finding Zeros on Two Sides of a Specified Point | |

| |

| |

| |

Using f zero to Find Several Roots | |

| |

| |

| |

Points of Discontinuity Mistaken for Roots | |

| |

| |

Problem Set | |

| |

| |

| |

Solution of Systems of Equations | |

| |

| |

| |

Linear Systems of Equations | |

| |

| |

| |

Numerical Solution of Linear Systems | |

| |

| |

| |

Gauss Elimination Method | |

| |

| |

| |

Choosing the Pivot Row: Partial Pivoting with Row Scaling | |

| |

| |

| |

Permutation Matrices | |

| |

| |

| |

Counting the Number of Operations | |

| |

| |

| |

Elimination | |

| |

| |

| |

Back Substitution | |

| |

| |

| |

Tridiagonal Systems | |

| |

| |

| |

Thomas Method | |

| |

| |

| |

MATLABï¿½ Built-in Function "\" | |

| |

| |

| |

LU Factorization Methods | |

| |

| |

| |

Doolittle Factorization | |

| |

| |

| |

Doolittle's Method to Solve a Linear System | |

| |

| |

| |

Operations Count | |

| |

| |

| |

Cholesky Factorization | |

| |

| |

| |

Cholesky's Method to Solve a Linear System | |

| |

| |

| |

Operations Count | |

| |

| |

| |

MATLABï¿½ Built-in Functions lu and chol | |

| |

| |

| |

Iterative Solution of Linear Systems | |

| |

| |

| |

Vector Norms | |

| |

| |

| |

Matrix Norms | |

| |

| |

| |

Compatibility of Vector and Matrix Norms | |

| |

| |

| |

General Iterative Method | |

| |

| |

| |

Convergence of the General Iterative Method | |

| |

| |

| |

Jacobi Iteration Method | |

| |

| |

| |

Convergence of the Jacobi Iteration Method | |

| |

| |

| |

Gauss-Seidel Iteration Method | |

| |

| |

| |

Convergence of the Gauss-Seidel Iteration Method | |

| |

| |

| |

Indirect Methods versus Direct Methods for Large Systems | |

| |

| |

| |

Ill-Conditioning and Error Analysis | |

| |

| |

| |

Condition Number | |

| |

| |

| |

Ill-Conditioning | |

| |

| |

| |

Indicators of Ill-Conditioning | |

| |

| |

| |

Computational Error | |

| |

| |

| |

Consequences of Ill-Conditioning | |

| |

| |

| |

Effects of Parameter Changes on the Solution | |

| |

| |

| |

Systems of NonLinear Equations | |

| |

| |

| |

Newton's Method for a System of Nonlinear Equations | |

| |

| |

| |

Method for Solving a System of Two Nonlinear Equations | |

| |

| |

| |

Method for Solving a System of n Nonlinear Equations | |

| |

| |

| |

Convergence of Newton's Method | |

| |

| |

| |

Fixed-Point Iteration Method for a System of Nonlinear Equations | |

| |

| |

| |

Convergence of the Fixed-Point Iteration Method | |

| |

| |

Problem Set | |

| |

| |

| |

Curve Fitting (Approximation) and Interpolation | |

| |

| |

| |

Least-Squares Regression | |

| |

| |

| |

Linear Regression | |

| |

| |

| |

Deciding a "Best" Fit Criterion | |

| |

| |

| |

Linear Least-Squares Regression | |

| |

| |

| |

Linearization of Nonlinear Data | |

| |

| |

| |

Exponential Function | |

| |

| |

| |

Power Function | |

| |

| |

| |

Saturation Function | |

| |

| |

| |

Polynomial Regression | |

| |

| |

| |

Quadratic Least-Squares Regression | |

| |

| |

| |

Cubic Least-Squares Regression | |

| |

| |

| |

MATLABï¿½ Built-in Functions polyfit and polyval | |

| |

| |

| |

Polynomial Interpolation | |

| |

| |

| |

Lagrange Interpolating Polynomials | |

| |

| |

| |

Drawbacks of Lagrange Interpolation | |

| |

| |

| |

Newton Divided-Difference Interpolating Polynomials | |

| |

| |

| |

Special Case: Equally Spaced Data | |

| |

| |

| |

Newton Forward-Difference Interpolating Polynomials | |

| |

| |

| |

Spline Interpolation | |

| |

| |

| |

Linear Splines | |

| |

| |

| |

Quadratic Splines | |

| |

| |

| |

Function Values at the Endpoints | |

| |

| |

| |

Function Values at the Interior Knots | |

| |

| |

| |

First Derivatives at the Interior Knots | |

| |

| |

| |

Second Derivative at the Left Endpoint is Zero | |

| |

| |

| |

Cubic Splines | |

| |

| |

| |

Construction of Cubic Splines: Clamped Boundary Conditions | |

| |

| |

| |

Construction of Cubic Splines: Free Boundary Conditions | |

| |

| |

| |

MATLABï¿½ Built-in Functions interp1 and spline | |

| |

| |

| |

Boundary Conditions | |

| |

| |

| |

Interactive Curve Fitting and Interpolation in MATLABï¿½ | |

| |

| |

| |

Fourier Approximation and Interpolation | |

| |

| |

| |

Sinusoidal Curve Fitting | |

| |

| |

| |

Linear Transformation of Data | |

| |

| |

| |

Discrete Fourier Transform | |

| |

| |

| |

Fast Fourier Transform | |

| |

| |

| |

Sande-Tukey Algorithm (N = 2<sup>p</sup>, p = integer) | |

| |

| |

| |

Cooley-Tukey Algorithm (N=2<sup>p</sup>, p = integer) | |

| |

| |

| |

MATLABï¿½ Built-in Function fft | |

| |

| |

| |

Interpolation Using fft | |

| |

| |

Problem Set | |

| |

| |

| |

Numerical Differentiation and Integration | |

| |

| |

| |

Numerical Differentiation | |

| |

| |

| |

Finite-Difference Formulas for Numerical Differentiation | |

| |

| |

| |

Finite-Difference Formulas for the First Derivative | |

| |

| |

| |

Two-Point Backward Difference Formula | |

| |

| |

| |

Two-Point Forward Difference Formula | |

| |

| |

| |

Two-Point Central Difference Formula | |

| |

| |

| |

Three-Point Backward Difference Formula | |

| |

| |

| |

Three-Point Forward Difference Formula | |

| |

| |

| |

Finite-Difference Formulas for the Second Derivative | |

| |

| |

| |

Three-Point Backward Difference Formula | |

| |

| |

| |

Three-Point Forward Difference Formula | |

| |

| |

| |

Three-Point Central Difference Formula | |

| |

| |

| |

Summary of Finite-Difference Formulas for First to Fourth Derivatives | |

| |

| |

| |

Estimate Improvement: Richardson's Extrapolation | |

| |

| |

| |

Derivative Estimates for Nonevenly Spaced Data | |

| |

| |

| |

MATLABï¿½ Built-in Functions dif f and polyder | |

| |

| |

| |

Numerical Integration: Newton-Cotes Formulas | |

| |

| |

| |

Newton-Cotes Formulas | |

| |

| |

| |

Rectangular Rule | |

| |

| |

| |

Composite Rectangular Rule | |

| |

| |

| |

Error Estimate for Composite Rectangular Rule | |

| |

| |

| |

Trapezoidal Rule | |

| |

| |

| |

Composite Trapezoidal Rule | |

| |

| |

| |

Error Estimate for Composite Trapezoidal Rule | |

| |

| |

| |

Simpson's Rules | |

| |

| |

| |

Simpson's 1/3 Rule | |

| |

| |

| |

Composite Simpson's 1/3 Rule | |

| |

| |

| |

Error Estimate for Composite Simpson's 1/3 Rule | |

| |

| |

| |

Simpson's 3/8 Rule | |

| |

| |

| |

Composite Simpson's 3/8 Rule | |

| |

| |

| |

Error Estimate for Composite Simpson's 3/8 Rule | |

| |

| |

| |

MATLABï¿½ Built-in Functions quad and trapz | |

| |

| |

| |

Numerical Integration of Analytical Functions: Romberg Integration, Gaussian Quadrature | |

| |

| |

| |

Richardson's Extrapolation, Romberg Integration | |

| |

| |

| |

Richardson's Extrapolation | |

| |

| |

| |

Romberg Integration | |

| |

| |

| |

Gaussian Quadrature | |

| |

| |

| |

Improper Integrals | |

| |

| |

Problem Set | |

| |

| |

| |

Numerical Solution of Initial-Value Problems | |

| |

| |

| |

One-Step Methods | |

| |

| |

| |

Euler's Method | |

| |

| |

| |

Error Analysis for Euler's Method | |

| |

| |

| |

Calculation of Local and Global Truncation Errors | |

| |

| |

| |

Higher-Order Taylor Methods | |

| |

| |

| |

Runge-Kutta Methods | |

| |

| |

| |

Second-Order Runge-Kutta Methods | |

| |

| |

| |

Improved Euler's Method | |

| |

| |

| |

Heun's Method | |

| |

| |

| |

Ralston's Method | |

| |

| |

| |

Graphical Representation of Heun's Method | |

| |

| |

| |

Third-Order Runge-Kutta Methods | |

| |

| |

| |

Classical RK3 Method | |

| |

| |

| |

Heun's RK3 Method | |

| |

| |

| |

Fourth-Order Runge-Kutta Methods | |

| |

| |

| |

Classical RK4 Method | |

| |

| |

| |

Higher-Order Runge-Kutta Methods | |

| |

| |

| |

Runge-Kutta-Fehlberg Method | |

| |

| |

| |

Multistep Methods | |

| |

| |

| |

Adams-Bashforth Method | |

| |

| |

| |

Second-Order Adams-Bashforth Formula | |

| |

| |

| |

Third-Order Adams-Bashforth Formula | |

| |

| |

| |

Fourth-Order Adams-Bashforth Formula | |

| |

| |

| |

Adams-Moulton Method | |

| |

| |

| |

Second-Order Adams-Moulton Formula | |

| |

| |

| |

Third-Order Adams-Moulton Formula | |

| |

| |

| |

Fourth-Order Adams-Moulton Formula | |

| |

| |

| |

Predictor-Corrector Methods | |

| |

| |

| |

Heun's Predictor-Corrector Method | |

| |

| |

| |

Adams-Bashforth-Moulton Predictor-Corrector Method | |

| |

| |

| |

Systems of Ordinary Differential Equations | |

| |

| |

| |

Transformation into a System of First-Order ODEs | |

| |

| |

| |

State Variables | |

| |

| |

| |

Notation | |

| |

| |

| |

State-Variable Equations | |

| |

| |

| |

Numerical Solution of a System of First-Order ODEs | |

| |

| |

| |

Euler's Method for Systems | |

| |

| |

| |

Heun's Method for Systems | |

| |

| |

| |

Classical RK4 Method for Systems | |

| |

| |

| |

Stability | |

| |

| |

| |

Euler's Method | |

| |

| |

| |

Euler's Implicit Method | |

| |

| |

| |

Stiff Differential Equations | |

| |

| |

| |

MATLABï¿½ Built-in Functions for Initial-Value Problems | |

| |

| |

| |

Nonstiff Equations | |

| |

| |

| |

Single First-Order IVP | |

| |

| |

| |

Setting ODE Solver Options | |

| |

| |

| |

System of First-Order IVPs | |

| |

| |

| |

Stiff Equations | |

| |

| |

Problem Set | |

| |

| |

| |

Numerical Solution of Boundary-Value Problems | |

| |

| |

| |

Shooting Method | |

| |

| |

| |

Finite-Difference Method | |

| |

| |

| |

BVPs with Mixed Boundary Conditions | |

| |

| |

| |

MATLABï¿½ Built-in Function bvp4c for BVPs | |

| |

| |

| |

Second-Order BVP | |

| |

| |

Problem Set | |

| |

| |

| |

Matrix Eigenvalue Problem | |

| |

| |

| |

Power Method: Estimation of the Dominant Eigenvalue | |

| |

| |

| |

Inverse Power Method: Estimation of the Smallest Eigenvalue | |

| |

| |

| |

Shifted Inverse Power Method: Estimation of the Eigenvalue Nearest a Specified Value | |

| |

| |

| |

Shifted Power Method | |

| |

| |

| |

MATLABï¿½ Built-in Function eig | |

| |

| |

| |

Deflation Methods | |

| |

| |

| |

Wielandt's Deflation Method | |

| |

| |

| |

Deflation Process | |

| |

| |

| |

Householder Tridiagonalization and QR Factorization Methods | |

| |

| |

| |

Householder's Tridiagonalization Method (Symmetric Matrices) | |

| |

| |

| |

Determination of Symmetric Orthogonal P<sub>k</sub> (k = 1,2,…, n-2) | |

| |

| |

| |

QR Factorization Method | |

| |

| |

| |

Determination of Q<sub>k</sub> and R<sub>k</sub> Matrices | |

| |

| |

| |

Structure of L<sub>k</sub> (k = 2,3,…, n) | |

| |

| |

| |

MATLABï¿½ Built-in Function qr | |

| |

| |

| |

Transformation to Hessenberg Form (Nonsymmetric Matrices) | |

| |

| |

Problem Set | |

| |

| |

| |

Numerical Solution of Partial Differential Equations | |

| |

| |

| |

Introduction | |

| |

| |

| |

Elliptic PDEs | |

| |

| |

| |

Dirichlet Problem | |

| |

| |

| |

Alternating Direction-Implicit Methods | |

| |

| |

| |

Peaceman-Rachford Alternating Direction-Implicit Method | |

| |

| |

| |

Neumann Problem | |

| |

| |

| |

Existence of Solution for Neumann Problem | |

| |

| |

| |

Mixed Problem | |

| |

| |

| |

More Complex Regions | |

| |

| |

| |

Parabolic PDEs | |

| |

| |

| |

Finite-Difference (FD) Method | |

| |

| |

| |

Stability and Convergence of the FD Method | |

| |

| |

| |

Crank-Nicolson (CN) Method | |

| |

| |

| |

CN Method versus FD Method | |

| |

| |

| |

Hyperbolic PDEs | |

| |

| |

| |

Starting the Procedure | |

| |

| |

Problem Set | |

| |

| |

Bibliography | |

| |

| |

Index | |