| |
| |
| Foreword | |
| |
| |
| Preface | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| An Illustrative Example | |
| |
| |
| |
| Regularization by Filtering | |
| |
| |
| |
| A Deterministic Error Analysis | |
| |
| |
| |
| Rates of Convergence | |
| |
| |
| |
| A Posteriori Regularization Parameter Selection | |
| |
| |
| |
| Variational Regularization Methods | |
| |
| |
| |
| Iterative Regularization Methods | |
| |
| |
| Exercises | |
| |
| |
| |
| Analytical Tools | |
| |
| |
| |
| Ill-Posedness and Regularization | |
| |
| |
| |
| Compact Operators, Singular Systems, and the SVD | |
| |
| |
| |
| Least Squares Solutions and the Pseudo-Inverse | |
| |
| |
| |
| Regularization Theory | |
| |
| |
| |
| Optimization Theory | |
| |
| |
| |
| Generalized Tikhonov Regularization | |
| |
| |
| |
| Penalty Functionals | |
| |
| |
| |
| Data Discrepancy Functionals | |
| |
| |
| |
| Some Analysis | |
| |
| |
| Exercises | |
| |
| |
| |
| Numerical Optimization Tools | |
| |
| |
| |
| The Steepest Descent Method | |
| |
| |
| |
| The Conjugate Gradient Method | |
| |
| |
| |
| Preconditioning | |
| |
| |
| |
| Nonlinear CG Method | |
| |
| |
| |
| Newton's Method | |
| |
| |
| |
| Trust Region Globalization of Newton's Method | |
| |
| |
| |
| The BFGS Method | |
| |
| |
| |
| Inexact Line Search | |
| |
| |
| Exercises | |
| |
| |
| |
| Statistical Estimation Theory | |
| |
| |
| |
| Preliminary Definitions and Notation | |
| |
| |
| |
| Maximum Likelihood Estimation | |
| |
| |
| |
| Bayesian Estimation | |
| |
| |
| |
| Linear Least Squares Estimation | |
| |
| |
| |
| Best Linear Unbiased Estimation | |
| |
| |
| |
| Minimum Variance Linear Estimation | |
| |
| |
| |
| The EM Algorithm | |
| |
| |
| |
| An Illustrative Example | |
| |
| |
| Exercises | |
| |
| |
| |
| Image Deblurring | |
| |
| |
| |
| A Mathematical Model for Image Blurring | |
| |
| |
| |
| A Two-Dimensional Test Problem | |
| |
| |
| |
| Computational Methods for Toeplitz Systems | |
| |
| |
| |
| Discrete Fourier Transform and Convolution | |
| |
| |
| |
| The FFT Algorithm | |
| |
| |
| |
| Toeplitz and Circulant Matrices | |
| |
| |
| |
| Best Circulant Approximation | |
| |
| |
| |
| Block Toeplitz and Block Circulant Matrices | |
| |
| |
| |
| Fourier-Based Deblurring Methods | |
| |
| |
| |
| Direct Fourier Inversion | |
| |
| |
| |
| CG for Block Toeplitz Systems | |
| |
| |
| |
| Block Circulant Preconditioners | |
| |
| |
| |
| A Comparison of Block Circulant Preconditioners | |
| |
| |
| |
| Multilevel Techniques | |
| |
| |
| Exercises | |
| |
| |
| |
| Parameter Identification | |
| |
| |
| |
| An Abstract Framework | |
| |
| |
| |
| Gradient Computations | |
| |
| |
| |
| Adjoint, or Costate, Methods | |
| |
| |
| |
| Hessian Computations | |
| |
| |
| |
| Gauss--Newton Hessian Approximation | |
| |
| |
| |
| A One-Dimensional Example | |
| |
| |
| |
| A Convergence Result | |
| |
| |
| Exercises | |
| |
| |
| |
| Regularization Parameter Selection Methods | |
| |
| |
| |
| The Unbiased Predictive Risk Estimator Method | |
| |
| |
| |
| Implementation of the UPRE Method | |
| |
| |
| |
| Randomized Trace Estimation | |
| |
| |
| |
| A Numerical Illustration of Trace Estimation | |
| |
| |
| |
| Nonlinear Variants of UPRE | |
| |
| |
| |
| Generalized Cross Validation | |
| |
| |
| |
| A Numerical Comparison of UPRE and GCV | |
| |
| |
| |
| The Discrepancy Principle | |
| |
| |
| |
| Implementation of the Discrepancy Principle | |
| |
| |
| |
| The L-Curve Method | |
| |
| |
| |
| A Numerical Illustration of the L-Curve Method | |
| |
| |
| |
| Other Regularization Parameter Selection Methods | |
| |
| |
| |
| Analysis of Regularization Parameter Selection Methods | |
| |
| |
| |
| Model Assumptions and Preliminary Results | |
| |
| |
| |
| Estimation and Predictive Errors for TSVD | |
| |
| |
| |
| Estimation and Predictive Errors for Tikhonov Regularization | |
| |
| |
| |
| Analysis of the Discrepancy Principle | |
| |
| |
| |
| Analysis of GCV | |
| |
| |
| |
| Analysis of the L-Curve Method | |
| |
| |
| |
| A Comparison of Methods | |
| |
| |
| Exercises | |
| |
| |
| |
| Total Variation Regularization | |
| |
| |
| |
| Motivation | |
| |
| |
| |
| Numerical Methods for Total Variation | |
| |
| |
| |
| A One-Dimensional Discretization | |
| |
| |
| |
| A Two-Dimensional Discretization | |
| |
| |
| |
| Steepest Descent and Newton's Method for Total Variation | |
| |
| |
| |
| Lagged Diffusivity Fixed Point Iteration | |
| |
| |
| |
| A Primal-Dual Newton Method | |
| |
| |
| |
| Other Methods | |
| |
| |
| |
| Numerical Comparisons | |
| |
| |
| |
| Results for a One-Dimensional Test Problem | |
| |
| |
| |
| Two-Dimensional Test Results | |
| |
| |
| |
| Mathematical Analysis of Total Variation | |
| |
| |
| |
| Approximations to the TV Functional | |
| |
| |
| Exercises | |
| |
| |
| |
| Nonnegativity Constraints | |
| |
| |
| |
| An Illustrative Example | |
| |
| |
| |
| Theory of Constrained Optimization | |
| |
| |
| |
| Nonnegativity Constraints | |
| |
| |
| |
| Numerical Methods for Nonnegatively Constrained Minimization | |
| |
| |
| |
| The Gradient Projection Method | |
| |
| |
| |
| A Projected Newton Method | |
| |
| |
| |
| A Gradient Projection-Reduced Newton Method | |
| |
| |
| |
| A Gradient Projection-CG Method | |
| |
| |
| |
| Other Methods | |
| |
| |
| |
| Numerical Test Results | |
| |
| |
| |
| Results for One-Dimensional Test Problems | |
| |
| |
| |
| Results for a Two-Dimensional Test Problem | |
| |
| |
| |
| Iterative Nonnegative Regularization Methods | |
| |
| |
| |
| Richardson--Lucy Iteration | |
| |
| |
| |
| A Modified Steepest Descent Algorithm | |
| |
| |
| Exercises | |
| |
| |
| Bibliography | |