| |
| |
| Preface | |
| |
| |
| Authors | |
| |
| |
| |
| Matrices, Matrix Algebra, and Elementary Matrix Operations | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Basic Concepts and Notation | |
| |
| |
| |
| Matrix and Vector Notation | |
| |
| |
| |
| Matrix Definition | |
| |
| |
| |
| Elementary Matrices | |
| |
| |
| |
| Elementary Matrix Operations | |
| |
| |
| |
| Matrix Algebra | |
| |
| |
| |
| Matrix Addition and Subtraction | |
| |
| |
| |
| Properties of Matrix Addition | |
| |
| |
| |
| Matrix Multiplication | |
| |
| |
| |
| Properties of Matrix Multiplication | |
| |
| |
| |
| Applications of Matrix Multiplication in Signal and Image Processing | |
| |
| |
| |
| Application in Linear Discrete One Dimensional Convolution | |
| |
| |
| |
| Application in Linear Discrete Two Dimensional Convolution | |
| |
| |
| |
| Matrix Representation of Discrete Fourier Transform | |
| |
| |
| |
| Elementary Row Operations | |
| |
| |
| |
| Row Echelon Form | |
| |
| |
| |
| Elementary Transformation Matrices | |
| |
| |
| |
| Type 1: Scaling Transformation Matrix (E<sub>1</sub> | |
| |
| |
| |
| Type 2: Interchange Transformation Matrix (E<sub>2</sub>) | |
| |
| |
| |
| Type 3: Combination Transformation Matrices (E<sub>3</sub>) | |
| |
| |
| |
| Solution of System of Linear Equations | |
| |
| |
| |
| Gaussian Elimination | |
| |
| |
| |
| Over Determined Systems | |
| |
| |
| |
| Under Determined Systems | |
| |
| |
| |
| Matrix Partitions | |
| |
| |
| |
| Column Partitions | |
| |
| |
| |
| Row Partitions | |
| |
| |
| |
| Block Multiplication | |
| |
| |
| |
| Inner, Outer, and Kronecker Products | |
| |
| |
| |
| Inner Product | |
| |
| |
| |
| Outer Product | |
| |
| |
| |
| Kronecker Products | |
| |
| |
| Problems | |
| |
| |
| |
| Determinants, Matrix Inversion and Solutions to Systems of Linear Equations | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Determinant of a Matrix | |
| |
| |
| |
| Properties of Determinant | |
| |
| |
| |
| Row Operations and Determinants | |
| |
| |
| |
| Interchange of Two Rows | |
| |
| |
| |
| Multiplying a Row of A by a Nonzero Constant | |
| |
| |
| |
| Adding a Multiple of One Row to Another Row | |
| |
| |
| |
| Singular Matrices | |
| |
| |
| |
| Matrix Inversion | |
| |
| |
| |
| Properties of Matrix Inversion | |
| |
| |
| |
| Gauss-Jordan Method for Calculating Inverse of a Matrix | |
| |
| |
| |
| Useful Formulas for Matrix Inversion | |
| |
| |
| |
| Recursive Least Square (RLS) Parameter Estimation | |
| |
| |
| |
| Solution of Simultaneous Linear Equations | |
| |
| |
| |
| Equivalent Systems | |
| |
| |
| |
| Strict Triangular Form | |
| |
| |
| |
| Cramer's Rule | |
| |
| |
| |
| LU Decomposition | |
| |
| |
| |
| Applications: Circuit Analysis | |
| |
| |
| |
| Homogeneous Coordinates System | |
| |
| |
| |
| Applications of Homogeneous Coordinates in Image Processing | |
| |
| |
| |
| Rank, Null Space and Invertibility of Matrices | |
| |
| |
| |
| Null Space N(A) | |
| |
| |
| |
| Column Space C(A) | |
| |
| |
| |
| Row Space R(A) | |
| |
| |
| |
| Rank of a Matrix | |
| |
| |
| |
| Special Matrices with Applications | |
| |
| |
| |
| Vandermonde Matrix | |
| |
| |
| |
| Hankel Matrix | |
| |
| |
| |
| Toeplitz Matrices | |
| |
| |
| |
| Permutation Matrix | |
| |
| |
| |
| Markov Matrices | |
| |
| |
| |
| Circulant Matrices | |
| |
| |
| |
| Hadamard Matrices | |
| |
| |
| |
| Nilpotent Matrices | |
| |
| |
| |
| Derivatives and Gradients | |
| |
| |
| |
| Derivative of Scalar with Respect to a Vector | |
| |
| |
| |
| Quadratic Functions | |
| |
| |
| |
| Derivative of a Vector Function with Respect to a Vector | |
| |
| |
| Problems | |
| |
| |
| |
| Linear Vector Spaces | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Linear Vector Space | |
| |
| |
| |
| Definition of Linear Vector Space | |
| |
| |
| |
| Examples of Linear Vector Spaces | |
| |
| |
| |
| Additional Properties of Linear Vector Spaces | |
| |
| |
| |
| Subspace of a Linear Vector Space | |
| |
| |
| |
| Span of a Set of Vectors | |
| |
| |
| |
| Spanning Set of a Vector Space | |
| |
| |
| |
| Linear Dependence | |
| |
| |
| |
| Basis Vectors | |
| |
| |
| |
| Change of Basis Vectors | |
| |
| |
| |
| Normed Vector Spaces | |
| |
| |
| |
| Definition of Normed Vector Space | |
| |
| |
| |
| Examples of Normed Vector Spaces | |
| |
| |
| |
| Distance Function | |
| |
| |
| |
| Equivalence of Norms | |
| |
| |
| |
| Inner Product Spaces | |
| |
| |
| |
| Definition of Inner Product | |
| |
| |
| |
| Examples of Inner Product Spaces | |
| |
| |
| |
| Schwarz's Inequality | |
| |
| |
| |
| Norm Derived from Inner Product | |
| |
| |
| |
| Applications of Schwarz Inequality in Communication Systems | |
| |
| |
| |
| Detection of a Discrete Signal �Buried� in White Noise | |
| |
| |
| |
| Detection of Continuous Signal �Buried� in Noise | |
| |
| |
| |
| Hilbert Space | |
| |
| |
| |
| Orthogonality | |
| |
| |
| |
| Orthonormal Set | |
| |
| |
| |
| Gram-Schmidt Orthogonalization Process | |
| |
| |
| |
| Orthogonal Matrices | |
| |
| |
| |
| Complete Orthonormal Set | |
| |
| |
| |
| Generalized Fourier Series (GFS) | |
| |
| |
| |
| Applications of GFS | |
| |
| |
| |
| Continuous Fourier Series | |
| |
| |
| |
| Discrete Fourier Transform (DFT) | |
| |
| |
| |
| Legendre Polynomial | |
| |
| |
| |
| Sinc Functions | |
| |
| |
| |
| Matrix Factorization | |
| |
| |
| |
| QR Factorization | |
| |
| |
| |
| Solution of Linear Equations Using QR Factorization | |
| |
| |
| Problems | |
| |
| |
| |
| Eigenvalues and Eigenvectors | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Matrices as Linear Transformations | |
| |
| |
| |
| Definition: Linear Transformation | |
| |
| |
| |
| Matrices as Linear Operators | |
| |
| |
| |
| Null Space of a Matrix | |
| |
| |
| |
| Projection Operator | |
| |
| |
| |
| Orthogonal Projection | |
| |
| |
| |
| Projection Theorem | |
| |
| |
| |
| Matrix Representation of Projection Operator | |
| |
| |
| |
| Eigenvalues and Eigenvectors | |
| |
| |
| |
| Definition of Eigenvalues and Eigenvectors | |
| |
| |
| |
| Properties of Eigenvalues and Eigenvectors | |
| |
| |
| |
| Independent Property | |
| |
| |
| |
| Product and Sum of Eigenvalues | |
| |
| |
| |
| Finding the Characteristic Polynomial of a Matrix | |
| |
| |
| |
| Modal Matrix | |
| |
| |
| |
| Matrix Diagonalization | |
| |
| |
| |
| Distinct Eigenvalues | |
| |
| |
| |
| Jordan Canonical Form | |
| |
| |
| |
| Special Matrices | |
| |
| |
| |
| Unitary Matrices | |
| |
| |
| |
| Hermitian Matrices | |
| |
| |
| |
| Definite Matrices | |
| |
| |
| |
| Positive Definite Matrices | |
| |
| |
| |
| Positive Semidefinite Matrices | |
| |
| |
| |
| Negative Definite Matrices | |
| |
| |
| |
| Negative Semidefinite Matrices | |
| |
| |
| |
| Test for Matrix Positiveness | |
| |
| |
| |
| Singular Value Decomposition (SVD) | |
| |
| |
| |
| Definition of SVD | |
| |
| |
| |
| Matrix Norm | |
| |
| |
| |
| Frobenius Norm | |
| |
| |
| |
| Matrix Condition Number | |
| |
| |
| |
| Numerical Computation of Eigenvalues and Eigenvectors | |
| |
| |
| |
| Power Method | |
| |
| |
| |
| Properties of Eigenvalues and Eigenvectors of Different Classes of Matrices | |
| |
| |
| |
| Applications | |
| |
| |
| |
| Image Edge Detection | |
| |
| |
| |
| Gradient Based Edge Detection of Gray Scale Images | |
| |
| |
| |
| Gradient Based Edge Detection of RGB Images | |
| |
| |
| |
| Vibration Analysis | |
| |
| |
| |
| Signal Subspace Decomposition | |
| |
| |
| |
| Frequency Estimation | |
| |
| |
| |
| Direction of Arrival Estimation | |
| |
| |
| Problems | |
| |
| |
| |
| Matrix Polynomials and Functions of Square Matrices | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Matrix Polynomials | |
| |
| |
| |
| Infinite Series of Matrices | |
| |
| |
| |
| Convergence of an Infinite Matrix Series | |
| |
| |
| |
| Cayley-Hamilton Theorem | |
| |
| |
| |
| Matrix Polynomial Reduction | |
| |
| |
| |
| Functions of Matrices | |
| |
| |
| |
| Sylvester's Expansion | |
| |
| |
| |
| Cayley-Hamilton Technique | |
| |
| |
| |
| Modal Matrix Technique | |
| |
| |
| |
| Special Matrix Functions | |
| |
| |
| |
| Matrix Exponential Function e<sup>At</sup> | |
| |
| |
| |
| Matrix Function A<sup>k</sup> | |
| |
| |
| |
| The State Space Modeling of Linear Continuous-time Systems | |
| |
| |
| |
| Concept of States | |
| |
| |
| |
| State Equations of Continuous Time Systems | |
| |
| |
| |
| State Space Representation of Continuous LTI Systems | |
| |
| |
| |
| Solution of Continuous-time State Space Equations | |
| |
| |
| |
| Solution of Homogenous State Equations and State Transition Matrix | |
| |
| |
| |
| Properties of State Transition Matrix | |
| |
| |
| |
| Computing State Transition Matrix | |
| |
| |
| |
| Complete Solution of State Equations | |
| |
| |
| |
| State Space Representation of Discrete-time Systems | |
| |
| |
| |
| Definition of States | |
| |
| |
| |
| State Equations | |
| |
| |
| |
| State Space Representation of Discrete-time LTI Systems | |
| |
| |
| |
| Solution of Discrete-time State Equations | |
| |
| |
| |
| Solution of Homogenous State Equation and State Transition Matrix | |
| |
| |
| |
| Properties of State Transition Matrix | |
| |
| |
| |
| Computing the State Transition Matrix | |
| |
| |
| |
| Complete Solution of the State Equations | |
| |
| |
| |
| Controllability of LTI Systems | |
| |
| |
| |
| Definition of Controllability | |
| |
| |
| |
| Controllability Condition | |
| |
| |
| |
| Observability of LTI Systems | |
| |
| |
| |
| Definition of Observability | |
| |
| |
| |
| Observability Condition | |
| |
| |
| Problems | |
| |
| |
| |
| Introduction to Optimization | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Stationary Points of Functions of Several Variables | |
| |
| |
| |
| Hessian Matrix | |
| |
| |
| |
| Least-Square (LS) Technique | |
| |
| |
| |
| LS Computation Using QR Factorization | |
| |
| |
| |
| LS Computation Using Singtilar Value Decomposition (SVD) | |
| |
| |
| |
| Weighted Least Square (WLS) | |
| |
| |
| |
| LS Curve Fitting | |
| |
| |
| |
| Applications of LS Technique | |
| |
| |
| |
| One Dimensional Wiener Filter | |
| |
| |
| |
| Choice of Q Matrix and Scale Factor � | |
| |
| |
| |
| Two Dimensional Wiener Filter | |
| |
| |
| |
| Total Least-Squares (TLS) | |
| |
| |
| |
| Eigen Filters | |
| |
| |
| |
| Stationary Points with Equality Constraints | |
| |
| |
| |
| Lagrange Multipliers | |
| |
| |
| |
| Applications | |
| |
| |
| |
| Maximum Entropy Problem | |
| |
| |
| |
| Design of Digital Finite Impulse Response (FIR) Filters | |
| |
| |
| Problems | |
| |
| |
| |
| The Laplace Transform | |
| |
| |
| |
| Definition of the Laplace Transform | |
| |
| |
| |
| The Inverse Laplace Transform | |
| |
| |
| |
| Partial Fraction Expansion | |
| |
| |
| |
| The z-Transform | |
| |
| |
| |
| Definition of the z-Transform | |
| |
| |
| |
| The Inverse z-Transform | |
| |
| |
| |
| Inversion by Partial Fraction Expansion | |
| |
| |
| Bibliography | |
| |
| |
| Index | |