| |

| |

Preface | |

| |

| |

Acknowledgments | |

| |

| |

Background and Preview | |

| |

| |

| |

The Filtering Problem | |

| |

| |

| |

Linear Optimum Filters | |

| |

| |

| |

Adaptive Filters | |

| |

| |

| |

Linear Filter Structures | |

| |

| |

| |

Approaches to the Development of Linear Adaptive Filters | |

| |

| |

| |

Adaptive Beamforming | |

| |

| |

| |

Four Classes of Applications | |

| |

| |

| |

Historical Notes | |

| |

| |

| |

Stochastic Processes and Models | |

| |

| |

| |

Partial Characterization of a Discrete-Time Stochastic Process | |

| |

| |

| |

Mean Ergodic Theorem | |

| |

| |

| |

Correlation Matrix | |

| |

| |

| |

Correlation Matrix of Sine Wave Plus Noise | |

| |

| |

| |

Stochastic Models | |

| |

| |

| |

Wold Decomposition | |

| |

| |

| |

Asymptotic Stationarity of an Autoregressive Process | |

| |

| |

| |

Yule-Walker Equations | |

| |

| |

| |

Computer Experiment: Autoregressive Process of Order Two | |

| |

| |

| |

Selecting the Model Order | |

| |

| |

| |

Complex Gaussian Processes | |

| |

| |

| |

Power Spectral Density | |

| |

| |

| |

Properties of Power Spectral Density | |

| |

| |

| |

Transmission of a Stationary Process Through a Linear Filter | |

| |

| |

| |

Cramï¿½r Spectral Representation for a Stationary Process | |

| |

| |

| |

Power Spectrum Estimation | |

| |

| |

| |

Other Statistical Characteristics of a Stochastic Process | |

| |

| |

| |

Polyspectra | |

| |

| |

| |

Spectral-Correlation Density | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

Wiener Filters | |

| |

| |

| |

Linear Optimum Filtering: Statement of the Problem | |

| |

| |

| |

Principle of Orthogonality | |

| |

| |

| |

Minimum Mean-Square Error | |

| |

| |

| |

Wiener-Hopf Equations | |

| |

| |

| |

Error-Performance Surface | |

| |

| |

| |

Multiple Linear Regression Model | |

| |

| |

| |

Example | |

| |

| |

| |

Linearly Constrained Minimum-Variance Filter | |

| |

| |

| |

Generalized Sidelobe Cancellers | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

Linear Prediction | |

| |

| |

| |

Forward Linear Prediction | |

| |

| |

| |

Backward Linear Prediction | |

| |

| |

| |

Levinson-Durbin Algorithm | |

| |

| |

| |

Properties of Prediction-Error Filters | |

| |

| |

| |

Schur-Cohn Test | |

| |

| |

| |

Autoregressive Modeling of a Stationary Stochastic Process | |

| |

| |

| |

Cholesky Factorization | |

| |

| |

| |

Lattice Predictors | |

| |

| |

| |

All-Pole, All-Pass Lattice Filter | |

| |

| |

| |

Joint-Process Estimation | |

| |

| |

| |

Predictive Modeling of Speech | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

Method of Steepest Descent | |

| |

| |

| |

Basic Idea of the Steepest-Descent Algorithm | |

| |

| |

| |

The Steepest-Descent Algorithm Applied to the Wiener Filter | |

| |

| |

| |

Stability of the Steepest-Descent Algorithm | |

| |

| |

| |

Example | |

| |

| |

| |

The Steepest-Descent Algorithm Viewed as a Deterministic Search Method | |

| |

| |

| |

Virtue and Limitation of the Steepest-Descent Algorithm | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

Method of Stochastic Gradient Descent | |

| |

| |

| |

Principles of Stochastic Gradient Descent | |

| |

| |

| |

Application 1: Least-Mean-Square (LMS) Algorithm | |

| |

| |

| |

Application 2: Gradient-Adaptive Lattice Filtering Algorithm | |

| |

| |

| |

Other Applications of Stochastic Gradient Descent | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

The Least-Mean-Square (LMS) Algorithm | |

| |

| |

| |

Signal-How Graph | |

| |

| |

| |

Optimality Considerations | |

| |

| |

| |

Applications | |

| |

| |

| |

Statistical Learning Theory | |

| |

| |

| |

Transient Behavior and Convergence Considerations | |

| |

| |

| |

Efficiency | |

| |

| |

| |

Computer Experiment on Adaptive Prediction | |

| |

| |

| |

Computer Experiment on Adaptive Equalization | |

| |

| |

| |

Computer Experiment on a Minimum-Variance Distortionless-Response Beamformer | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

Normalized Least-Mean-Square (LMS) Algorithm and Its Generalization | |

| |

| |

| |

Normalized LMS Algorithm: The Solution to a Constrained Optimization Problem | |

| |

| |

| |

Stability of the Normalized LMS Algorithm | |

| |

| |

| |

Step-Size Control for Acoustic Echo Cancellation | |

| |

| |

| |

Geometric Considerations Pertaining to the Convergence Process for Real-Valued Data | |

| |

| |

| |

Affine Projection Adaptive Filters | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

Block-Adaptive Filters | |

| |

| |

| |

Block-Adaptive Filters: Basic Ideas | |

| |

| |

| |

Fast Block LMS Algorithm | |

| |

| |

| |

Unconstrained Frequency-Domain Adaptive Filters | |

| |

| |

| |

Self-Orthogonalizing Adaptive Filters | |

| |

| |

| |

Computer Experiment on Adaptive Equalization | |

| |

| |

| |

Subband Adaptive Filters | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

Method of Least-Squares | |

| |

| |

| |

Statement of the Linear Least-Squares Estimation Problem | |

| |

| |

| |

Data Windowing | |

| |

| |

| |

Principle of Orthogonality Revisited | |

| |

| |

| |

Minimum Sum of Error Squares | |

| |

| |

| |

Normal Equations and Linear Least-Squares Filters | |

| |

| |

| |

Time-Average Correlation Matrix ï¿½ | |

| |

| |

| |

Reformulation of the Normal Equations in Terms of Data Matrices | |

| |

| |

| |

Properties of Least-Squares Estimates | |

| |

| |

| |

Minimum-Variance Distortionless Response (MVDR) Spectrum Estimation | |

| |

| |

| |

Regularized MVDR Beamforming | |

| |

| |

| |

Singular-Value Decomposition | |

| |

| |

| |

Pseudoinverse | |

| |

| |

| |

Interpretation of Singular Values and Singular Vectors | |

| |

| |

| |

Minimum-Norm Solution to the Linear Least-Squares Problem | |

| |

| |

| |

Normalized LMS Algorithm Viewed as the Minimum-Norm Solution to an Underdetermined Least-Squares Estimation Problem | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

The Recursive Least-Squares (RLS) Algorithm | |

| |

| |

| |

Some Preliminaries | |

| |

| |

| |

The Matrix Inversion Lemma | |

| |

| |

| |

The Exponentially Weighted RLS Algorithm | |

| |

| |

| |

Selection of the Regularization Parameter | |

| |

| |

| |

Updated Recursion for the Sum of Weighted Error Squares | |

| |

| |

| |

Example: Single-Weight Adaptive Noise Canceller | |

| |

| |

| |

Statistical Learning Theory | |

| |

| |

| |

Efficiency | |

| |

| |

| |

Computer Experiment on Adaptive Equalization | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

Robustness | |

| |

| |

| |

Robustness, Adaptation, and Disturbances | |

| |

| |

| |

Robustness: Preliminary Considerations Rooted in H<sup>∞</sup> Optimization | |

| |

| |

| |

Robustness of the LMS Algorithm | |

| |

| |

| |

Robustness of the RLS Algorithm | |

| |

| |

| |

Comparative Evaluations of the LMS and RLS Algorithms from the Perspective of Robustness | |

| |

| |

| |

Risk-Sensitive Optimality | |

| |

| |

| |

Trade-Offs Between Robustness and Efficiency | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

Finite-Precision Effects | |

| |

| |

| |

Quantization Errors | |

| |

| |

| |

Least-Mean-Square (LMS) Algorithm | |

| |

| |

| |

Recursive Least-Squares (RLS) Algorithm | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

Adaptation in Nonstationary Environments | |

| |

| |

| |

Causes and Consequences of Nonstationarity | |

| |

| |

| |

The System Identification Problem | |

| |

| |

| |

Degree of Nonstationarity | |

| |

| |

| |

Criteria for Tracking Assessment | |

| |

| |

| |

Tracking Performance of the LMS Algorithm | |

| |

| |

| |

Tracking Performance of the RLS Algorithm | |

| |

| |

| |

Comparison of the Tracking Performance of LMS and RLS Algorithms | |

| |

| |

| |

Tuning of Adaptation Parameters | |

| |

| |

| |

Incremental Delta-Bar-Delta (IDBD) Algorithm | |

| |

| |

| |

Autostep Method | |

| |

| |

| |

Computer Experiment: Mixture of Stationary and Nonstationary Environmental Data | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

Kalman Filters | |

| |

| |

| |

Recursive Minimum Mean-Square Estimation for Scalar Random Variables | |

| |

| |

| |

Statement of the Kalman Filtering Problem | |

| |

| |

| |

The Innovations Process | |

| |

| |

| |

Estimation of the State Using the Innovations Process | |

| |

| |

| |

Filtering | |

| |

| |

| |

Initial Conditions | |

| |

| |

| |

Summary of the Kalman Filter | |

| |

| |

| |

Optimality Criteria for Kalman Filtering | |

| |

| |

| |

Kalman Filter as the Unifying Basis for RLS Algorithms | |

| |

| |

| |

Covariance Filtering Algorithm | |

| |

| |

| |

Information Filtering Algorithm | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

Square-Root Adaptive Filtering Algorithms | |

| |

| |

| |

Square-Root Kalman Filters | |

| |

| |

| |

Building Square-Root Adaptive Filters on the Two Kalman Filter Variants | |

| |

| |

| |

QRD-RLS Algorithm | |

| |

| |

| |

Adaptive Beamforming | |

| |

| |

| |

Inverse QRD-RLS Algorithm | |

| |

| |

| |

Finite-Precision Effects | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

Order-Recursive Adaptive Filtering Algorithm | |

| |

| |

| |

Order-Recursive Adaptive Filters Using Least-Squares Estimation: An Overview | |

| |

| |

| |

Adaptive Forward Linear Prediction | |

| |

| |

| |

Adaptive Backward Linear Prediction | |

| |

| |

| |

Conversion Factor | |

| |

| |

| |

Least-Squares Lattice (LSL) Predictor | |

| |

| |

| |

Angle-Normalized Estimation Errors | |

| |

| |

| |

First-Order State-Space Models for Lattice Filtering | |

| |

| |

| |

QR-Decomposition-Based Least-Squares Lattice (QRD-LSL) Filters | |

| |

| |

| |

Fundamental Properties of the QRD-LSL Filter | |

| |

| |

| |

Computer Experiment on Adaptive Equalization | |

| |

| |

| |

Recursive (LSL) Filters Using A Posteriori Estimation Errors | |

| |

| |

| |

Recursive LSL Filters Using A Priori Estimation Errors with Error Feedback | |

| |

| |

| |

Relation Between Recursive LSL and RLS Algorithms | |

| |

| |

| |

Finite-Precision Effects | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

| |

Blind Deconvolution | |

| |

| |

| |

Overview of Blind Deconvolution | |

| |

| |

| |

Channel Identifiability Using Cyclostationary Statistics | |

| |

| |

| |

Subspace Decomposition for Fractionally Spaced Blind Identification | |

| |

| |

| |

Bussgang Algorithm for Blind Equalization | |

| |

| |

| |

Extension of the Bussgang Algorithm to Complex Baseband Channels | |

| |

| |

| |

Special Cases of the Bussgang Algorithm | |

| |

| |

| |

Fractionally Spaced Bussgang Equalizers | |

| |

| |

| |

Estimation of Unknown Probability Distribution Function of Signal Source | |

| |

| |

| |

Summary and Discussion | |

| |

| |

Problems | |

| |

| |

Epilogue | |

| |

| |

| |

Robustness, Efficiency, and Complexity | |

| |

| |

| |

Kernel-Based Nonlinear Adaptive Filtering | |

| |

| |

| |

Theory of Complex Variables | |

| |

| |

| |

Cauchy-Riemann Equations | |

| |

| |

| |

Cauchy's Integral Formula | |

| |

| |

| |

Laurent's Series | |

| |

| |

| |

Singularities and Residues | |

| |

| |

| |

Cauchy's Residue Theorem | |

| |

| |

| |

Principle of the Argument | |

| |

| |

| |

Inversion Integral for the z-Transform | |

| |

| |

| |

Parseval's Theorem | |

| |

| |

| |

Wirtinger Calculus for Computing Complex Gradients | |

| |

| |

| |

Wirtinger Calculus: Scalar Gradients | |

| |

| |

| |

Generalized Wirtinger Calculus: Gradient Vectors | |

| |

| |

| |

Another Approach to Compute Gradient Vectors | |

| |

| |

| |

Expressions for the Partial Derivatives ∂f/∂z and ∂f/∂z* | |

| |

| |

| |

Method of Lagrange Multipliers | |

| |

| |

| |

Optimization Involving a Single Equality Constraint | |

| |

| |

| |

Optimization Involving Multiple Equality Constraints | |

| |

| |

| |

Optimum Beamformer | |

| |

| |

| |

Estimation Theory | |

| |

| |

| |

Likelihood Function | |

| |

| |

| |

Cramï¿½r-Rao Inequality | |

| |

| |

| |

Properties of Maximum-Likelihood Estimators | |

| |

| |

| |

Conditional Mean Estimator | |

| |

| |

| |

Eigenanalysis | |

| |

| |

| |

The Eigenvalue Problem | |

| |

| |

| |

Properties of Eigenvalues and Eigenvectors | |

| |

| |

| |

Low-Rank Modeling | |

| |

| |

| |

Eigenfilters | |

| |

| |

| |

Eigenvalue Computations | |

| |

| |

| |

Langevin Equation of Nonequilibrium Thermodynamics | |

| |

| |

| |

Brownian Motion | |

| |

| |

| |

Langevin Equation | |

| |

| |

| |

Rotations and Reflections | |

| |

| |

| |

Plane Rotations | |

| |

| |

| |

Two-Sided Jacobi Algorithm | |

| |

| |

| |

Cyclic Jacobi Algorithm | |

| |

| |

| |

Householder Transformation | |

| |

| |

| |

The QR Algorithm | |

| |

| |

| |

Complex Wishart Distribution | |

| |

| |

| |

Definition | |

| |

| |

| |

The Chi-Square Distribution as a Special Case | |

| |

| |

| |

Properties of the Complex Wishart Distribution | |

| |

| |

| |

Expectation of the Inverse Correlation Matrix ï¿½<sup>-1</sup>(n) | |

| |

| |

Glossary | |

| |

| |

Text Conventions | |

| |

| |

Abbreviations | |

| |

| |

Principal Symbols | |

| |

| |

Bibliography | |

| |

| |

Suggested Reading | |

| |

| |

Index | |