| |
| |
| Preface | |
| |
| |
| |
| Vectors of Random Variables | |
| |
| |
| |
| Notation | |
| |
| |
| |
| Statistical Models | |
| |
| |
| |
| Linear Regression Models | |
| |
| |
| |
| Expectation and Covariance Operators | |
| |
| |
| Exercises 1a | |
| |
| |
| |
| Mean and Variance of Quadratic Forms | |
| |
| |
| Exercises 1b | |
| |
| |
| |
| Moment Generating Functions and Independence | |
| |
| |
| Exercises 1c | |
| |
| |
| Miscellaneous Exercises 1 | |
| |
| |
| |
| Multivariate Normal Distribution | |
| |
| |
| |
| Density Function | |
| |
| |
| Exercises 2a | |
| |
| |
| |
| Moment Generating Functions | |
| |
| |
| Exercises 2b | |
| |
| |
| |
| Statistical Independence | |
| |
| |
| Exercises 2c | |
| |
| |
| |
| Distribution of Quadratic Forms | |
| |
| |
| Exercises 2d | |
| |
| |
| Miscellaneous Exercises 2 | |
| |
| |
| |
| Linear Regression: Estimation and Distribution Theory | |
| |
| |
| |
| Least Squares Estimation | |
| |
| |
| Exercises 3a | |
| |
| |
| |
| Properties of Least Squares Estimates | |
| |
| |
| Exercises 3b | |
| |
| |
| |
| Unbiased Estimation of [sigma superscript 2] | |
| |
| |
| Exercises 3c | |
| |
| |
| |
| Distribution Theory | |
| |
| |
| Exercises 3d | |
| |
| |
| |
| Maximum Likelihood Estimation | |
| |
| |
| |
| Orthogonal Columns in the Regression Matrix | |
| |
| |
| Exercises 3e | |
| |
| |
| |
| Introducing Further Explanatory Variables | |
| |
| |
| |
| General Theory | |
| |
| |
| |
| One Extra Variable | |
| |
| |
| Exercises 3f | |
| |
| |
| |
| Estimation with Linear Restrictions | |
| |
| |
| |
| Method of Lagrange Multipliers | |
| |
| |
| |
| Method of Orthogonal Projections | |
| |
| |
| Exercises 3g | |
| |
| |
| |
| Design Matrix of Less Than Full Rank | |
| |
| |
| |
| Least Squares Estimation | |
| |
| |
| Exercises 3h | |
| |
| |
| |
| Estimable Functions | |
| |
| |
| Exercises 3i | |
| |
| |
| |
| Introducing Further Explanatory Variables | |
| |
| |
| |
| Introducing Linear Restrictions | |
| |
| |
| Exercises 3j | |
| |
| |
| |
| Generalized Least Squares | |
| |
| |
| Exercises 3k | |
| |
| |
| |
| Centering and Scaling the Explanatory Variables | |
| |
| |
| |
| Centering | |
| |
| |
| |
| Scaling | |
| |
| |
| Exercises 3l | |
| |
| |
| |
| Bayesian Estimation | |
| |
| |
| Exercises 3m | |
| |
| |
| |
| Robust Regression | |
| |
| |
| |
| M-Estimates | |
| |
| |
| |
| Estimates Based on Robust Location and Scale Measures | |
| |
| |
| |
| Measuring Robustness | |
| |
| |
| |
| Other Robust Estimates | |
| |
| |
| Exercises 3n | |
| |
| |
| Miscellaneous Exercises 3 | |
| |
| |
| |
| Hypothesis Testing | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Likelihood Ratio Test | |
| |
| |
| |
| F-Test | |
| |
| |
| |
| Motivation | |
| |
| |
| |
| Derivation | |
| |
| |
| Exercises 4a | |
| |
| |
| |
| Some Examples | |
| |
| |
| |
| The Straight Line | |
| |
| |
| Exercises 4b | |
| |
| |
| |
| Multiple Correlation Coefficient | |
| |
| |
| Exercises 4c | |
| |
| |
| |
| Canonical Form for H | |
| |
| |
| Exercises 4d | |
| |
| |
| |
| Goodness-of-Fit Test | |
| |
| |
| |
| F-Test and Projection Matrices | |
| |
| |
| Miscellaneous Exercises 4 | |
| |
| |
| |
| Confidence Intervals and Regions | |
| |
| |
| |
| Simultaneous Interval Estimation | |
| |
| |
| |
| Simultaneous Inferences | |
| |
| |
| |
| Comparison of Methods | |
| |
| |
| |
| Confidence Regions | |
| |
| |
| |
| Hypothesis Testing and Confidence Intervals | |
| |
| |
| |
| Confidence Bands for the Regression Surface | |
| |
| |
| |
| Confidence Intervals | |
| |
| |
| |
| Confidence Bands | |
| |
| |
| |
| Prediction Intervals and Bands for the Response | |
| |
| |
| |
| Prediction Intervals | |
| |
| |
| |
| Simultaneous Prediction Bands | |
| |
| |
| |
| Enlarging the Regression Matrix | |
| |
| |
| Miscellaneous Exercises 5 | |
| |
| |
| |
| Straight-Line Regression | |
| |
| |
| |
| The Straight Line | |
| |
| |
| |
| Confidence Intervals for the Slope and Intercept | |
| |
| |
| |
| Confidence Interval for the x-Intercept | |
| |
| |
| |
| Prediction Intervals and Bands | |
| |
| |
| |
| Prediction Intervals for the Response | |
| |
| |
| |
| Inverse Prediction (Calibration) | |
| |
| |
| Exercises 6a | |
| |
| |
| |
| Straight Line through the Origin | |
| |
| |
| |
| Weighted Least Squares for the Straight Line | |
| |
| |
| |
| Known Weights | |
| |
| |
| |
| Unknown Weights | |
| |
| |
| Exercises 6b | |
| |
| |
| |
| Comparing Straight Lines | |
| |
| |
| |
| General Model | |
| |
| |
| |
| Use of Dummy Explanatory Variables | |
| |
| |
| Exercises 6c | |
| |
| |
| |
| Two-Phase Linear Regression | |
| |
| |
| |
| Local Linear Regression | |
| |
| |
| Miscellaneous Exercises 6 | |
| |
| |
| |
| Polynomial Regression | |
| |
| |
| |
| Polynomials in One Variable | |
| |
| |
| |
| Problem of Ill-Conditioning | |
| |
| |
| |
| Using Orthogonal Polynomials | |
| |
| |
| |
| Controlled Calibration | |
| |
| |
| |
| Piecewise Polynomial Fitting | |
| |
| |
| |
| Unsatisfactory Fit | |
| |
| |
| |
| Spline Functions | |
| |
| |
| |
| Smoothing Splines | |
| |
| |
| |
| Polynomial Regression in Several Variables | |
| |
| |
| |
| Response Surfaces | |
| |
| |
| |
| Multidimensional Smoothing | |
| |
| |
| Miscellaneous Exercises 7 | |
| |
| |
| |
| Analysis of Variance | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| One-Way Classification | |
| |
| |
| |
| General Theory | |
| |
| |
| |
| Confidence Intervals | |
| |
| |
| |
| Underlying Assumptions | |
| |
| |
| Exercises 8a | |
| |
| |
| |
| Two-Way Classification (Unbalanced) | |
| |
| |
| |
| Representation as a Regression Model | |
| |
| |
| |
| Hypothesis Testing | |
| |
| |
| |
| Procedures for Testing the Hypotheses | |
| |
| |
| |
| Confidence Intervals | |
| |
| |
| Exercises 8b | |
| |
| |
| |
| Two-Way Classification (Balanced) | |
| |
| |
| Exercises 8c | |
| |
| |
| |
| Two-Way Classification (One Observation per Mean) | |
| |
| |
| |
| Underlying Assumptions | |
| |
| |
| |
| Higher-Way Classifications with Equal Numbers per Mean | |
| |
| |
| |
| Definition of Interactions | |
| |
| |
| |
| Hypothesis Testing | |
| |
| |
| |
| Missing Observations | |
| |
| |
| Exercises 8d | |
| |
| |
| |
| Designs with Simple Block Structure | |
| |
| |
| |
| Analysis of Covariance | |
| |
| |
| Exercises 8e | |
| |
| |
| Miscellaneous Exercises 8 | |
| |
| |
| |
| Departures from Underlying Assumptions | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Bias | |
| |
| |
| |
| Bias Due to Underfitting | |
| |
| |
| |
| Bias Due to Overfitting | |
| |
| |
| Exercises 9a | |
| |
| |
| |
| Incorrect Variance Matrix | |
| |
| |
| Exercises 9b | |
| |
| |
| |
| Effect of Outliers | |
| |
| |
| |
| Robustness of the F-Test to Nonnormality | |
| |
| |
| |
| Effect of the Regressor Variables | |
| |
| |
| |
| Quadratically Balanced F-Tests | |
| |
| |
| Exercises 9c | |
| |
| |
| |
| Effect of Random Explanatory Variables | |
| |
| |
| |
| Random Explanatory Variables Measured without Error | |
| |
| |
| |
| Fixed Explanatory Variables Measured with Error | |
| |
| |
| |
| Round-off Errors | |
| |
| |
| |
| Some Working Rules | |
| |
| |
| |
| Random Explanatory Variables Measured with Error | |
| |
| |
| |
| Controlled Variables Model | |
| |
| |
| |
| Collinearity | |
| |
| |
| |
| Effect on the Variances of the Estimated Coefficients | |
| |
| |
| |
| Variance Inflation Factors | |
| |
| |
| |
| Variances and Eigenvalues | |
| |
| |
| |
| Perturbation Theory | |
| |
| |
| |
| Collinearity and Prediction | |
| |
| |
| Exercises 9d | |
| |
| |
| Miscellaneous Exercises 9 | |
| |
| |
| |
| Departures from Assumptions: Diagnosis and Remedies | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Residuals and Hat Matrix Diagonals | |
| |
| |
| Exercises 10a | |
| |
| |
| |
| Dealing with Curvature | |
| |
| |
| |
| Visualizing Regression Surfaces | |
| |
| |
| |
| Transforming to Remove Curvature | |
| |
| |
| |
| Adding and Deleting Variables | |
| |
| |
| Exercises 10b | |
| |
| |
| |
| Nonconstant Variance and Serial Correlation | |
| |
| |
| |
| Detecting Nonconstant Variance | |
| |
| |
| |
| Estimating Variance Functions | |
| |
| |
| |
| Transforming to Equalize Variances | |
| |
| |
| |
| Serial Correlation and the Durbin-Watson Test | |
| |
| |
| Exercises 10c | |
| |
| |
| |
| Departures from Normality | |
| |
| |
| |
| Normal Plotting | |
| |
| |
| |
| Transforming the Response | |
| |
| |
| |
| Transforming Both Sides | |
| |
| |
| Exercises 10d | |
| |
| |
| |
| Detecting and Dealing with Outliers | |
| |
| |
| |
| Types of Outliers | |
| |
| |
| |
| Identifying High-Leverage Points | |
| |
| |
| |
| Leave-One-Out Case Diagnostics | |
| |
| |
| |
| Test for Outliers | |
| |
| |
| |
| Other Methods | |
| |
| |
| Exercises 10e | |
| |
| |
| |
| Diagnosing Collinearity | |
| |
| |
| |
| Drawbacks of Centering | |
| |
| |
| |
| Detection of Points Influencing Collinearity | |
| |
| |
| |
| Remedies for Collinearity | |
| |
| |
| Exercises 10f | |
| |
| |
| Miscellaneous Exercises 10 | |
| |
| |
| |
| Computational Algorithms for Fitting a Regression | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Basic Methods | |
| |
| |
| |
| Direct Solution of the Normal Equations | |
| |
| |
| |
| Calculation of the Matrix X'X | |
| |
| |
| |
| Solving the Normal Equations | |
| |
| |
| Exercises 11a | |
| |
| |
| |
| QR Decomposition | |
| |
| |
| |
| Calculation of Regression Quantities | |
| |
| |
| |
| Algorithms for the QR and WU Decompositions | |
| |
| |
| Exercises 11b | |
| |
| |
| |
| Singular Value Decomposition | |
| |
| |
| |
| Regression Calculations Using the SVD | |
| |
| |
| |
| Computing the SVD | |
| |
| |
| |
| Weighted Least Squares | |
| |
| |
| |
| Adding and Deleting Cases and Variables | |
| |
| |
| |
| Updating Formulas | |
| |
| |
| |
| Connection with the Sweep Operator | |
| |
| |
| |
| Adding and Deleting Cases and Variables Using QR | |
| |
| |
| |
| Centering the Data | |
| |
| |
| |
| Comparing Methods | |
| |
| |
| |
| Resources | |
| |
| |
| |
| Efficiency | |
| |
| |
| |
| Accuracy | |
| |
| |
| |
| Two Examples | |
| |
| |
| |
| Summary | |
| |
| |
| Exercises 11c | |
| |
| |
| |
| Rank-Deficient Case | |
| |
| |
| |
| Modifying the QR Decomposition | |
| |
| |
| |
| Solving the Least Squares Problem | |
| |
| |
| |
| Calculating Rank in the Presence of Round-off Error | |
| |
| |
| |
| Using the Singular Value Decomposition | |
| |
| |
| |
| Computing the Hat Matrix Diagonals | |
| |
| |
| |
| Using the Cholesky Factorization | |
| |
| |
| |
| Using the Thin QR Decomposition | |
| |
| |
| |
| Calculating Test Statistics | |
| |
| |
| |
| Robust Regression Calculations | |
| |
| |
| |
| Algorithms for L[subscript 1] Regression | |
| |
| |
| |
| Algorithms for M- and GM-Estimation | |
| |
| |
| |
| Elemental Regressions | |
| |
| |
| |
| Algorithms for High-Breakdown Methods | |
| |
| |
| Exercises 11d | |
| |
| |
| Miscellaneous Exercises 11 | |
| |
| |
| |
| Prediction and Model Selection | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Why Select? | |
| |
| |
| Exercises 12a | |
| |
| |
| |
| Choosing the Best Subset | |
| |
| |
| |
| Goodness-of-Fit Criteria | |
| |
| |
| |
| Criteria Based on Prediction Error | |
| |
| |
| |
| Estimating Distributional Discrepancies | |
| |
| |
| |
| Approximating Posterior Probabilities | |
| |
| |
| Exercises 12b | |
| |
| |
| |
| Stepwise Methods | |
| |
| |
| |
| Forward Selection | |
| |
| |
| |
| Backward Elimination | |
| |
| |
| |
| Stepwise Regression | |
| |
| |
| Exercises 12c | |
| |
| |
| |
| Shrinkage Methods | |
| |
| |
| |
| Stein Shrinkage | |
| |
| |
| |
| Ridge Regression | |
| |
| |
| |
| Garrote and Lasso Estimates | |
| |
| |
| Exercises 12d | |
| |
| |
| |
| Bayesian Methods | |
| |
| |
| |
| Predictive Densities | |
| |
| |
| |
| Bayesian Prediction | |
| |
| |
| |
| Bayesian Model Averaging | |
| |
| |
| Exercises 12e | |
| |
| |
| |
| Effect of Model Selection on Inference | |
| |
| |
| |
| Conditional and Unconditional Distributions | |
| |
| |
| |
| Bias | |
| |
| |
| |
| Conditional Means and Variances | |
| |
| |
| |
| Estimating Coefficients Using Conditional Likelihood | |
| |
| |
| |
| Other Effects of Model Selection | |
| |
| |
| Exercises 12f | |
| |
| |
| |
| Computational Considerations | |
| |
| |
| |
| Methods for All Possible Subsets | |
| |
| |
| |
| Generating the Best Regressions | |
| |
| |
| |
| All Possible Regressions Using QR Decompositions | |
| |
| |
| Exercises 12g | |
| |
| |
| |
| Comparison of Methods | |
| |
| |
| |
| Identifying the Correct Subset | |
| |
| |
| |
| Using Prediction Error as a Criterion | |
| |
| |
| Exercises 12h | |
| |
| |
| Miscellaneous Exercises 12 | |
| |
| |
| |
| Some Matrix Algebra | |
| |
| |
| |
| Trace and Eigenvalues | |
| |
| |
| |
| Rank | |
| |
| |
| |
| Positive-Semidefinite Matrices | |
| |
| |
| |
| Positive-Definite Matrices | |
| |
| |
| |
| Permutation Matrices | |
| |
| |
| |
| Idempotent Matrices | |
| |
| |
| |
| Eigenvalue Applications | |
| |
| |
| |
| Vector Differentiation | |
| |
| |
| |
| Patterned Matrices | |
| |
| |
| |
| Generalized Inverse | |
| |
| |
| |
| Some Useful Results | |
| |
| |
| |
| Singular Value Decomposition | |
| |
| |
| |
| Some Miscellaneous Statistical Results | |
| |
| |
| |
| Fisher Scoring | |
| |
| |
| |
| Orthogonal Projections | |
| |
| |
| |
| Orthogonal Decomposition of Vectors | |
| |
| |
| |
| Orthogonal Complements | |
| |
| |
| |
| Projections on Subspaces | |
| |
| |
| |
| Tables | |
| |
| |
| |
| Percentage Points of the Bonferroni t-Statistic | |
| |
| |
| |
| Distribution of the Largest Absolute Value of k Student t Variables | |
| |
| |
| |
| Working-Hotelling Confidence Bands for Finite Intervals | |
| |
| |
| Outline Solutions to Selected Exercises | |
| |
| |
| References | |
| |
| |
| Index | |