| |

| |

Preface | |

| |

| |

Authors | |

| |

| |

| |

Matrices, Matrix Algebra, and Elementary Matrix Operations | |

| |

| |

| |

Introduction | |

| |

| |

| |

Basic Concepts and Notation | |

| |

| |

| |

Matrix and Vector Notation | |

| |

| |

| |

Matrix Definition | |

| |

| |

| |

Elementary Matrices | |

| |

| |

| |

Elementary Matrix Operations | |

| |

| |

| |

Matrix Algebra | |

| |

| |

| |

Matrix Addition and Subtraction | |

| |

| |

| |

Properties of Matrix Addition | |

| |

| |

| |

Matrix Multiplication | |

| |

| |

| |

Properties of Matrix Multiplication | |

| |

| |

| |

Applications of Matrix Multiplication in Signal and Image Processing | |

| |

| |

| |

Application in Linear Discrete One Dimensional Convolution | |

| |

| |

| |

Application in Linear Discrete Two Dimensional Convolution | |

| |

| |

| |

Matrix Representation of Discrete Fourier Transform | |

| |

| |

| |

Elementary Row Operations | |

| |

| |

| |

Row Echelon Form | |

| |

| |

| |

Elementary Transformation Matrices | |

| |

| |

| |

Type 1: Scaling Transformation Matrix (E<sub>1</sub> | |

| |

| |

| |

Type 2: Interchange Transformation Matrix (E<sub>2</sub>) | |

| |

| |

| |

Type 3: Combination Transformation Matrices (E<sub>3</sub>) | |

| |

| |

| |

Solution of System of Linear Equations | |

| |

| |

| |

Gaussian Elimination | |

| |

| |

| |

Over Determined Systems | |

| |

| |

| |

Under Determined Systems | |

| |

| |

| |

Matrix Partitions | |

| |

| |

| |

Column Partitions | |

| |

| |

| |

Row Partitions | |

| |

| |

| |

Block Multiplication | |

| |

| |

| |

Inner, Outer, and Kronecker Products | |

| |

| |

| |

Inner Product | |

| |

| |

| |

Outer Product | |

| |

| |

| |

Kronecker Products | |

| |

| |

Problems | |

| |

| |

| |

Determinants, Matrix Inversion and Solutions to Systems of Linear Equations | |

| |

| |

| |

Introduction | |

| |

| |

| |

Determinant of a Matrix | |

| |

| |

| |

Properties of Determinant | |

| |

| |

| |

Row Operations and Determinants | |

| |

| |

| |

Interchange of Two Rows | |

| |

| |

| |

Multiplying a Row of A by a Nonzero Constant | |

| |

| |

| |

Adding a Multiple of One Row to Another Row | |

| |

| |

| |

Singular Matrices | |

| |

| |

| |

Matrix Inversion | |

| |

| |

| |

Properties of Matrix Inversion | |

| |

| |

| |

Gauss-Jordan Method for Calculating Inverse of a Matrix | |

| |

| |

| |

Useful Formulas for Matrix Inversion | |

| |

| |

| |

Recursive Least Square (RLS) Parameter Estimation | |

| |

| |

| |

Solution of Simultaneous Linear Equations | |

| |

| |

| |

Equivalent Systems | |

| |

| |

| |

Strict Triangular Form | |

| |

| |

| |

Cramer's Rule | |

| |

| |

| |

LU Decomposition | |

| |

| |

| |

Applications: Circuit Analysis | |

| |

| |

| |

Homogeneous Coordinates System | |

| |

| |

| |

Applications of Homogeneous Coordinates in Image Processing | |

| |

| |

| |

Rank, Null Space and Invertibility of Matrices | |

| |

| |

| |

Null Space N(A) | |

| |

| |

| |

Column Space C(A) | |

| |

| |

| |

Row Space R(A) | |

| |

| |

| |

Rank of a Matrix | |

| |

| |

| |

Special Matrices with Applications | |

| |

| |

| |

Vandermonde Matrix | |

| |

| |

| |

Hankel Matrix | |

| |

| |

| |

Toeplitz Matrices | |

| |

| |

| |

Permutation Matrix | |

| |

| |

| |

Markov Matrices | |

| |

| |

| |

Circulant Matrices | |

| |

| |

| |

Hadamard Matrices | |

| |

| |

| |

Nilpotent Matrices | |

| |

| |

| |

Derivatives and Gradients | |

| |

| |

| |

Derivative of Scalar with Respect to a Vector | |

| |

| |

| |

Quadratic Functions | |

| |

| |

| |

Derivative of a Vector Function with Respect to a Vector | |

| |

| |

Problems | |

| |

| |

| |

Linear Vector Spaces | |

| |

| |

| |

Introduction | |

| |

| |

| |

Linear Vector Space | |

| |

| |

| |

Definition of Linear Vector Space | |

| |

| |

| |

Examples of Linear Vector Spaces | |

| |

| |

| |

Additional Properties of Linear Vector Spaces | |

| |

| |

| |

Subspace of a Linear Vector Space | |

| |

| |

| |

Span of a Set of Vectors | |

| |

| |

| |

Spanning Set of a Vector Space | |

| |

| |

| |

Linear Dependence | |

| |

| |

| |

Basis Vectors | |

| |

| |

| |

Change of Basis Vectors | |

| |

| |

| |

Normed Vector Spaces | |

| |

| |

| |

Definition of Normed Vector Space | |

| |

| |

| |

Examples of Normed Vector Spaces | |

| |

| |

| |

Distance Function | |

| |

| |

| |

Equivalence of Norms | |

| |

| |

| |

Inner Product Spaces | |

| |

| |

| |

Definition of Inner Product | |

| |

| |

| |

Examples of Inner Product Spaces | |

| |

| |

| |

Schwarz's Inequality | |

| |

| |

| |

Norm Derived from Inner Product | |

| |

| |

| |

Applications of Schwarz Inequality in Communication Systems | |

| |

| |

| |

Detection of a Discrete Signal ï¿½Buriedï¿½ in White Noise | |

| |

| |

| |

Detection of Continuous Signal ï¿½Buriedï¿½ in Noise | |

| |

| |

| |

Hilbert Space | |

| |

| |

| |

Orthogonality | |

| |

| |

| |

Orthonormal Set | |

| |

| |

| |

Gram-Schmidt Orthogonalization Process | |

| |

| |

| |

Orthogonal Matrices | |

| |

| |

| |

Complete Orthonormal Set | |

| |

| |

| |

Generalized Fourier Series (GFS) | |

| |

| |

| |

Applications of GFS | |

| |

| |

| |

Continuous Fourier Series | |

| |

| |

| |

Discrete Fourier Transform (DFT) | |

| |

| |

| |

Legendre Polynomial | |

| |

| |

| |

Sinc Functions | |

| |

| |

| |

Matrix Factorization | |

| |

| |

| |

QR Factorization | |

| |

| |

| |

Solution of Linear Equations Using QR Factorization | |

| |

| |

Problems | |

| |

| |

| |

Eigenvalues and Eigenvectors | |

| |

| |

| |

Introduction | |

| |

| |

| |

Matrices as Linear Transformations | |

| |

| |

| |

Definition: Linear Transformation | |

| |

| |

| |

Matrices as Linear Operators | |

| |

| |

| |

Null Space of a Matrix | |

| |

| |

| |

Projection Operator | |

| |

| |

| |

Orthogonal Projection | |

| |

| |

| |

Projection Theorem | |

| |

| |

| |

Matrix Representation of Projection Operator | |

| |

| |

| |

Eigenvalues and Eigenvectors | |

| |

| |

| |

Definition of Eigenvalues and Eigenvectors | |

| |

| |

| |

Properties of Eigenvalues and Eigenvectors | |

| |

| |

| |

Independent Property | |

| |

| |

| |

Product and Sum of Eigenvalues | |

| |

| |

| |

Finding the Characteristic Polynomial of a Matrix | |

| |

| |

| |

Modal Matrix | |

| |

| |

| |

Matrix Diagonalization | |

| |

| |

| |

Distinct Eigenvalues | |

| |

| |

| |

Jordan Canonical Form | |

| |

| |

| |

Special Matrices | |

| |

| |

| |

Unitary Matrices | |

| |

| |

| |

Hermitian Matrices | |

| |

| |

| |

Definite Matrices | |

| |

| |

| |

Positive Definite Matrices | |

| |

| |

| |

Positive Semidefinite Matrices | |

| |

| |

| |

Negative Definite Matrices | |

| |

| |

| |

Negative Semidefinite Matrices | |

| |

| |

| |

Test for Matrix Positiveness | |

| |

| |

| |

Singular Value Decomposition (SVD) | |

| |

| |

| |

Definition of SVD | |

| |

| |

| |

Matrix Norm | |

| |

| |

| |

Frobenius Norm | |

| |

| |

| |

Matrix Condition Number | |

| |

| |

| |

Numerical Computation of Eigenvalues and Eigenvectors | |

| |

| |

| |

Power Method | |

| |

| |

| |

Properties of Eigenvalues and Eigenvectors of Different Classes of Matrices | |

| |

| |

| |

Applications | |

| |

| |

| |

Image Edge Detection | |

| |

| |

| |

Gradient Based Edge Detection of Gray Scale Images | |

| |

| |

| |

Gradient Based Edge Detection of RGB Images | |

| |

| |

| |

Vibration Analysis | |

| |

| |

| |

Signal Subspace Decomposition | |

| |

| |

| |

Frequency Estimation | |

| |

| |

| |

Direction of Arrival Estimation | |

| |

| |

Problems | |

| |

| |

| |

Matrix Polynomials and Functions of Square Matrices | |

| |

| |

| |

Introduction | |

| |

| |

| |

Matrix Polynomials | |

| |

| |

| |

Infinite Series of Matrices | |

| |

| |

| |

Convergence of an Infinite Matrix Series | |

| |

| |

| |

Cayley-Hamilton Theorem | |

| |

| |

| |

Matrix Polynomial Reduction | |

| |

| |

| |

Functions of Matrices | |

| |

| |

| |

Sylvester's Expansion | |

| |

| |

| |

Cayley-Hamilton Technique | |

| |

| |

| |

Modal Matrix Technique | |

| |

| |

| |

Special Matrix Functions | |

| |

| |

| |

Matrix Exponential Function e<sup>At</sup> | |

| |

| |

| |

Matrix Function A<sup>k</sup> | |

| |

| |

| |

The State Space Modeling of Linear Continuous-time Systems | |

| |

| |

| |

Concept of States | |

| |

| |

| |

State Equations of Continuous Time Systems | |

| |

| |

| |

State Space Representation of Continuous LTI Systems | |

| |

| |

| |

Solution of Continuous-time State Space Equations | |

| |

| |

| |

Solution of Homogenous State Equations and State Transition Matrix | |

| |

| |

| |

Properties of State Transition Matrix | |

| |

| |

| |

Computing State Transition Matrix | |

| |

| |

| |

Complete Solution of State Equations | |

| |

| |

| |

State Space Representation of Discrete-time Systems | |

| |

| |

| |

Definition of States | |

| |

| |

| |

State Equations | |

| |

| |

| |

State Space Representation of Discrete-time LTI Systems | |

| |

| |

| |

Solution of Discrete-time State Equations | |

| |

| |

| |

Solution of Homogenous State Equation and State Transition Matrix | |

| |

| |

| |

Properties of State Transition Matrix | |

| |

| |

| |

Computing the State Transition Matrix | |

| |

| |

| |

Complete Solution of the State Equations | |

| |

| |

| |

Controllability of LTI Systems | |

| |

| |

| |

Definition of Controllability | |

| |

| |

| |

Controllability Condition | |

| |

| |

| |

Observability of LTI Systems | |

| |

| |

| |

Definition of Observability | |

| |

| |

| |

Observability Condition | |

| |

| |

Problems | |

| |

| |

| |

Introduction to Optimization | |

| |

| |

| |

Introduction | |

| |

| |

| |

Stationary Points of Functions of Several Variables | |

| |

| |

| |

Hessian Matrix | |

| |

| |

| |

Least-Square (LS) Technique | |

| |

| |

| |

LS Computation Using QR Factorization | |

| |

| |

| |

LS Computation Using Singtilar Value Decomposition (SVD) | |

| |

| |

| |

Weighted Least Square (WLS) | |

| |

| |

| |

LS Curve Fitting | |

| |

| |

| |

Applications of LS Technique | |

| |

| |

| |

One Dimensional Wiener Filter | |

| |

| |

| |

Choice of Q Matrix and Scale Factor ï¿½ | |

| |

| |

| |

Two Dimensional Wiener Filter | |

| |

| |

| |

Total Least-Squares (TLS) | |

| |

| |

| |

Eigen Filters | |

| |

| |

| |

Stationary Points with Equality Constraints | |

| |

| |

| |

Lagrange Multipliers | |

| |

| |

| |

Applications | |

| |

| |

| |

Maximum Entropy Problem | |

| |

| |

| |

Design of Digital Finite Impulse Response (FIR) Filters | |

| |

| |

Problems | |

| |

| |

| |

The Laplace Transform | |

| |

| |

| |

Definition of the Laplace Transform | |

| |

| |

| |

The Inverse Laplace Transform | |

| |

| |

| |

Partial Fraction Expansion | |

| |

| |

| |

The z-Transform | |

| |

| |

| |

Definition of the z-Transform | |

| |

| |

| |

The Inverse z-Transform | |

| |

| |

| |

Inversion by Partial Fraction Expansion | |

| |

| |

Bibliography | |

| |

| |

Index | |