| |

| |

Preface | |

| |

| |

| |

Vectors of Random Variables | |

| |

| |

| |

Notation | |

| |

| |

| |

Statistical Models | |

| |

| |

| |

Linear Regression Models | |

| |

| |

| |

Expectation and Covariance Operators | |

| |

| |

Exercises 1a | |

| |

| |

| |

Mean and Variance of Quadratic Forms | |

| |

| |

Exercises 1b | |

| |

| |

| |

Moment Generating Functions and Independence | |

| |

| |

Exercises 1c | |

| |

| |

Miscellaneous Exercises 1 | |

| |

| |

| |

Multivariate Normal Distribution | |

| |

| |

| |

Density Function | |

| |

| |

Exercises 2a | |

| |

| |

| |

Moment Generating Functions | |

| |

| |

Exercises 2b | |

| |

| |

| |

Statistical Independence | |

| |

| |

Exercises 2c | |

| |

| |

| |

Distribution of Quadratic Forms | |

| |

| |

Exercises 2d | |

| |

| |

Miscellaneous Exercises 2 | |

| |

| |

| |

Linear Regression: Estimation and Distribution Theory | |

| |

| |

| |

Least Squares Estimation | |

| |

| |

Exercises 3a | |

| |

| |

| |

Properties of Least Squares Estimates | |

| |

| |

Exercises 3b | |

| |

| |

| |

Unbiased Estimation of [sigma superscript 2] | |

| |

| |

Exercises 3c | |

| |

| |

| |

Distribution Theory | |

| |

| |

Exercises 3d | |

| |

| |

| |

Maximum Likelihood Estimation | |

| |

| |

| |

Orthogonal Columns in the Regression Matrix | |

| |

| |

Exercises 3e | |

| |

| |

| |

Introducing Further Explanatory Variables | |

| |

| |

| |

General Theory | |

| |

| |

| |

One Extra Variable | |

| |

| |

Exercises 3f | |

| |

| |

| |

Estimation with Linear Restrictions | |

| |

| |

| |

Method of Lagrange Multipliers | |

| |

| |

| |

Method of Orthogonal Projections | |

| |

| |

Exercises 3g | |

| |

| |

| |

Design Matrix of Less Than Full Rank | |

| |

| |

| |

Least Squares Estimation | |

| |

| |

Exercises 3h | |

| |

| |

| |

Estimable Functions | |

| |

| |

Exercises 3i | |

| |

| |

| |

Introducing Further Explanatory Variables | |

| |

| |

| |

Introducing Linear Restrictions | |

| |

| |

Exercises 3j | |

| |

| |

| |

Generalized Least Squares | |

| |

| |

Exercises 3k | |

| |

| |

| |

Centering and Scaling the Explanatory Variables | |

| |

| |

| |

Centering | |

| |

| |

| |

Scaling | |

| |

| |

Exercises 3l | |

| |

| |

| |

Bayesian Estimation | |

| |

| |

Exercises 3m | |

| |

| |

| |

Robust Regression | |

| |

| |

| |

M-Estimates | |

| |

| |

| |

Estimates Based on Robust Location and Scale Measures | |

| |

| |

| |

Measuring Robustness | |

| |

| |

| |

Other Robust Estimates | |

| |

| |

Exercises 3n | |

| |

| |

Miscellaneous Exercises 3 | |

| |

| |

| |

Hypothesis Testing | |

| |

| |

| |

Introduction | |

| |

| |

| |

Likelihood Ratio Test | |

| |

| |

| |

F-Test | |

| |

| |

| |

Motivation | |

| |

| |

| |

Derivation | |

| |

| |

Exercises 4a | |

| |

| |

| |

Some Examples | |

| |

| |

| |

The Straight Line | |

| |

| |

Exercises 4b | |

| |

| |

| |

Multiple Correlation Coefficient | |

| |

| |

Exercises 4c | |

| |

| |

| |

Canonical Form for H | |

| |

| |

Exercises 4d | |

| |

| |

| |

Goodness-of-Fit Test | |

| |

| |

| |

F-Test and Projection Matrices | |

| |

| |

Miscellaneous Exercises 4 | |

| |

| |

| |

Confidence Intervals and Regions | |

| |

| |

| |

Simultaneous Interval Estimation | |

| |

| |

| |

Simultaneous Inferences | |

| |

| |

| |

Comparison of Methods | |

| |

| |

| |

Confidence Regions | |

| |

| |

| |

Hypothesis Testing and Confidence Intervals | |

| |

| |

| |

Confidence Bands for the Regression Surface | |

| |

| |

| |

Confidence Intervals | |

| |

| |

| |

Confidence Bands | |

| |

| |

| |

Prediction Intervals and Bands for the Response | |

| |

| |

| |

Prediction Intervals | |

| |

| |

| |

Simultaneous Prediction Bands | |

| |

| |

| |

Enlarging the Regression Matrix | |

| |

| |

Miscellaneous Exercises 5 | |

| |

| |

| |

Straight-Line Regression | |

| |

| |

| |

The Straight Line | |

| |

| |

| |

Confidence Intervals for the Slope and Intercept | |

| |

| |

| |

Confidence Interval for the x-Intercept | |

| |

| |

| |

Prediction Intervals and Bands | |

| |

| |

| |

Prediction Intervals for the Response | |

| |

| |

| |

Inverse Prediction (Calibration) | |

| |

| |

Exercises 6a | |

| |

| |

| |

Straight Line through the Origin | |

| |

| |

| |

Weighted Least Squares for the Straight Line | |

| |

| |

| |

Known Weights | |

| |

| |

| |

Unknown Weights | |

| |

| |

Exercises 6b | |

| |

| |

| |

Comparing Straight Lines | |

| |

| |

| |

General Model | |

| |

| |

| |

Use of Dummy Explanatory Variables | |

| |

| |

Exercises 6c | |

| |

| |

| |

Two-Phase Linear Regression | |

| |

| |

| |

Local Linear Regression | |

| |

| |

Miscellaneous Exercises 6 | |

| |

| |

| |

Polynomial Regression | |

| |

| |

| |

Polynomials in One Variable | |

| |

| |

| |

Problem of Ill-Conditioning | |

| |

| |

| |

Using Orthogonal Polynomials | |

| |

| |

| |

Controlled Calibration | |

| |

| |

| |

Piecewise Polynomial Fitting | |

| |

| |

| |

Unsatisfactory Fit | |

| |

| |

| |

Spline Functions | |

| |

| |

| |

Smoothing Splines | |

| |

| |

| |

Polynomial Regression in Several Variables | |

| |

| |

| |

Response Surfaces | |

| |

| |

| |

Multidimensional Smoothing | |

| |

| |

Miscellaneous Exercises 7 | |

| |

| |

| |

Analysis of Variance | |

| |

| |

| |

Introduction | |

| |

| |

| |

One-Way Classification | |

| |

| |

| |

General Theory | |

| |

| |

| |

Confidence Intervals | |

| |

| |

| |

Underlying Assumptions | |

| |

| |

Exercises 8a | |

| |

| |

| |

Two-Way Classification (Unbalanced) | |

| |

| |

| |

Representation as a Regression Model | |

| |

| |

| |

Hypothesis Testing | |

| |

| |

| |

Procedures for Testing the Hypotheses | |

| |

| |

| |

Confidence Intervals | |

| |

| |

Exercises 8b | |

| |

| |

| |

Two-Way Classification (Balanced) | |

| |

| |

Exercises 8c | |

| |

| |

| |

Two-Way Classification (One Observation per Mean) | |

| |

| |

| |

Underlying Assumptions | |

| |

| |

| |

Higher-Way Classifications with Equal Numbers per Mean | |

| |

| |

| |

Definition of Interactions | |

| |

| |

| |

Hypothesis Testing | |

| |

| |

| |

Missing Observations | |

| |

| |

Exercises 8d | |

| |

| |

| |

Designs with Simple Block Structure | |

| |

| |

| |

Analysis of Covariance | |

| |

| |

Exercises 8e | |

| |

| |

Miscellaneous Exercises 8 | |

| |

| |

| |

Departures from Underlying Assumptions | |

| |

| |

| |

Introduction | |

| |

| |

| |

Bias | |

| |

| |

| |

Bias Due to Underfitting | |

| |

| |

| |

Bias Due to Overfitting | |

| |

| |

Exercises 9a | |

| |

| |

| |

Incorrect Variance Matrix | |

| |

| |

Exercises 9b | |

| |

| |

| |

Effect of Outliers | |

| |

| |

| |

Robustness of the F-Test to Nonnormality | |

| |

| |

| |

Effect of the Regressor Variables | |

| |

| |

| |

Quadratically Balanced F-Tests | |

| |

| |

Exercises 9c | |

| |

| |

| |

Effect of Random Explanatory Variables | |

| |

| |

| |

Random Explanatory Variables Measured without Error | |

| |

| |

| |

Fixed Explanatory Variables Measured with Error | |

| |

| |

| |

Round-off Errors | |

| |

| |

| |

Some Working Rules | |

| |

| |

| |

Random Explanatory Variables Measured with Error | |

| |

| |

| |

Controlled Variables Model | |

| |

| |

| |

Collinearity | |

| |

| |

| |

Effect on the Variances of the Estimated Coefficients | |

| |

| |

| |

Variance Inflation Factors | |

| |

| |

| |

Variances and Eigenvalues | |

| |

| |

| |

Perturbation Theory | |

| |

| |

| |

Collinearity and Prediction | |

| |

| |

Exercises 9d | |

| |

| |

Miscellaneous Exercises 9 | |

| |

| |

| |

Departures from Assumptions: Diagnosis and Remedies | |

| |

| |

| |

Introduction | |

| |

| |

| |

Residuals and Hat Matrix Diagonals | |

| |

| |

Exercises 10a | |

| |

| |

| |

Dealing with Curvature | |

| |

| |

| |

Visualizing Regression Surfaces | |

| |

| |

| |

Transforming to Remove Curvature | |

| |

| |

| |

Adding and Deleting Variables | |

| |

| |

Exercises 10b | |

| |

| |

| |

Nonconstant Variance and Serial Correlation | |

| |

| |

| |

Detecting Nonconstant Variance | |

| |

| |

| |

Estimating Variance Functions | |

| |

| |

| |

Transforming to Equalize Variances | |

| |

| |

| |

Serial Correlation and the Durbin-Watson Test | |

| |

| |

Exercises 10c | |

| |

| |

| |

Departures from Normality | |

| |

| |

| |

Normal Plotting | |

| |

| |

| |

Transforming the Response | |

| |

| |

| |

Transforming Both Sides | |

| |

| |

Exercises 10d | |

| |

| |

| |

Detecting and Dealing with Outliers | |

| |

| |

| |

Types of Outliers | |

| |

| |

| |

Identifying High-Leverage Points | |

| |

| |

| |

Leave-One-Out Case Diagnostics | |

| |

| |

| |

Test for Outliers | |

| |

| |

| |

Other Methods | |

| |

| |

Exercises 10e | |

| |

| |

| |

Diagnosing Collinearity | |

| |

| |

| |

Drawbacks of Centering | |

| |

| |

| |

Detection of Points Influencing Collinearity | |

| |

| |

| |

Remedies for Collinearity | |

| |

| |

Exercises 10f | |

| |

| |

Miscellaneous Exercises 10 | |

| |

| |

| |

Computational Algorithms for Fitting a Regression | |

| |

| |

| |

Introduction | |

| |

| |

| |

Basic Methods | |

| |

| |

| |

Direct Solution of the Normal Equations | |

| |

| |

| |

Calculation of the Matrix X'X | |

| |

| |

| |

Solving the Normal Equations | |

| |

| |

Exercises 11a | |

| |

| |

| |

QR Decomposition | |

| |

| |

| |

Calculation of Regression Quantities | |

| |

| |

| |

Algorithms for the QR and WU Decompositions | |

| |

| |

Exercises 11b | |

| |

| |

| |

Singular Value Decomposition | |

| |

| |

| |

Regression Calculations Using the SVD | |

| |

| |

| |

Computing the SVD | |

| |

| |

| |

Weighted Least Squares | |

| |

| |

| |

Adding and Deleting Cases and Variables | |

| |

| |

| |

Updating Formulas | |

| |

| |

| |

Connection with the Sweep Operator | |

| |

| |

| |

Adding and Deleting Cases and Variables Using QR | |

| |

| |

| |

Centering the Data | |

| |

| |

| |

Comparing Methods | |

| |

| |

| |

Resources | |

| |

| |

| |

Efficiency | |

| |

| |

| |

Accuracy | |

| |

| |

| |

Two Examples | |

| |

| |

| |

Summary | |

| |

| |

Exercises 11c | |

| |

| |

| |

Rank-Deficient Case | |

| |

| |

| |

Modifying the QR Decomposition | |

| |

| |

| |

Solving the Least Squares Problem | |

| |

| |

| |

Calculating Rank in the Presence of Round-off Error | |

| |

| |

| |

Using the Singular Value Decomposition | |

| |

| |

| |

Computing the Hat Matrix Diagonals | |

| |

| |

| |

Using the Cholesky Factorization | |

| |

| |

| |

Using the Thin QR Decomposition | |

| |

| |

| |

Calculating Test Statistics | |

| |

| |

| |

Robust Regression Calculations | |

| |

| |

| |

Algorithms for L[subscript 1] Regression | |

| |

| |

| |

Algorithms for M- and GM-Estimation | |

| |

| |

| |

Elemental Regressions | |

| |

| |

| |

Algorithms for High-Breakdown Methods | |

| |

| |

Exercises 11d | |

| |

| |

Miscellaneous Exercises 11 | |

| |

| |

| |

Prediction and Model Selection | |

| |

| |

| |

Introduction | |

| |

| |

| |

Why Select? | |

| |

| |

Exercises 12a | |

| |

| |

| |

Choosing the Best Subset | |

| |

| |

| |

Goodness-of-Fit Criteria | |

| |

| |

| |

Criteria Based on Prediction Error | |

| |

| |

| |

Estimating Distributional Discrepancies | |

| |

| |

| |

Approximating Posterior Probabilities | |

| |

| |

Exercises 12b | |

| |

| |

| |

Stepwise Methods | |

| |

| |

| |

Forward Selection | |

| |

| |

| |

Backward Elimination | |

| |

| |

| |

Stepwise Regression | |

| |

| |

Exercises 12c | |

| |

| |

| |

Shrinkage Methods | |

| |

| |

| |

Stein Shrinkage | |

| |

| |

| |

Ridge Regression | |

| |

| |

| |

Garrote and Lasso Estimates | |

| |

| |

Exercises 12d | |

| |

| |

| |

Bayesian Methods | |

| |

| |

| |

Predictive Densities | |

| |

| |

| |

Bayesian Prediction | |

| |

| |

| |

Bayesian Model Averaging | |

| |

| |

Exercises 12e | |

| |

| |

| |

Effect of Model Selection on Inference | |

| |

| |

| |

Conditional and Unconditional Distributions | |

| |

| |

| |

Bias | |

| |

| |

| |

Conditional Means and Variances | |

| |

| |

| |

Estimating Coefficients Using Conditional Likelihood | |

| |

| |

| |

Other Effects of Model Selection | |

| |

| |

Exercises 12f | |

| |

| |

| |

Computational Considerations | |

| |

| |

| |

Methods for All Possible Subsets | |

| |

| |

| |

Generating the Best Regressions | |

| |

| |

| |

All Possible Regressions Using QR Decompositions | |

| |

| |

Exercises 12g | |

| |

| |

| |

Comparison of Methods | |

| |

| |

| |

Identifying the Correct Subset | |

| |

| |

| |

Using Prediction Error as a Criterion | |

| |

| |

Exercises 12h | |

| |

| |

Miscellaneous Exercises 12 | |

| |

| |

| |

Some Matrix Algebra | |

| |

| |

| |

Trace and Eigenvalues | |

| |

| |

| |

Rank | |

| |

| |

| |

Positive-Semidefinite Matrices | |

| |

| |

| |

Positive-Definite Matrices | |

| |

| |

| |

Permutation Matrices | |

| |

| |

| |

Idempotent Matrices | |

| |

| |

| |

Eigenvalue Applications | |

| |

| |

| |

Vector Differentiation | |

| |

| |

| |

Patterned Matrices | |

| |

| |

| |

Generalized Inverse | |

| |

| |

| |

Some Useful Results | |

| |

| |

| |

Singular Value Decomposition | |

| |

| |

| |

Some Miscellaneous Statistical Results | |

| |

| |

| |

Fisher Scoring | |

| |

| |

| |

Orthogonal Projections | |

| |

| |

| |

Orthogonal Decomposition of Vectors | |

| |

| |

| |

Orthogonal Complements | |

| |

| |

| |

Projections on Subspaces | |

| |

| |

| |

Tables | |

| |

| |

| |

Percentage Points of the Bonferroni t-Statistic | |

| |

| |

| |

Distribution of the Largest Absolute Value of k Student t Variables | |

| |

| |

| |

Working-Hotelling Confidence Bands for Finite Intervals | |

| |

| |

Outline Solutions to Selected Exercises | |

| |

| |

References | |

| |

| |

Index | |