| |
| |
Preface to the First Edition | |
| |
| |
Preface to the Second Edition | |
| |
| |
Preface to the Third Edition | |
| |
| |
| |
Introduction | |
| |
| |
| |
Linear Models and Regression Analysis | |
| |
| |
| |
Plan of the Book | |
| |
| |
| |
The Simple Linear Regression Model | |
| |
| |
| |
The Linear Model | |
| |
| |
| |
Least Squares Estimation | |
| |
| |
| |
Direct Regression Method | |
| |
| |
| |
Properties of the Direct Regression Estimators | |
| |
| |
| |
Centered Model | |
| |
| |
| |
No Intercept Term Model | |
| |
| |
| |
Maximum Likelihood Estimation | |
| |
| |
| |
Testing of Hypotheses and Confidence Interval Estimation | |
| |
| |
| |
Analysis of Variance | |
| |
| |
| |
Goodness of Fit of Regression | |
| |
| |
| |
Reverse Regression Method | |
| |
| |
| |
Orthogonal Regression Method | |
| |
| |
| |
Reduced Major Axis Regression Method | |
| |
| |
| |
Least Absolute Deviation Regression Method | |
| |
| |
| |
Estimation of Parameters when X Is Stochastic | |
| |
| |
| |
The Multiple Linear Regression Model and Its Extensions | |
| |
| |
| |
The Linear Model | |
| |
| |
| |
The Principle of Ordinary Least Squares (OLS) | |
| |
| |
| |
Geometric Properties of OLS | |
| |
| |
| |
Best Linear Unbiased Estimation | |
| |
| |
| |
Basic Theorems | |
| |
| |
| |
Linear Estimators | |
| |
| |
| |
Mean Dispersion Error | |
| |
| |
| |
Estimation (Prediction) of the Error Term � and �<sup>2</sup> | |
| |
| |
| |
Classical Regression under Normal Errors | |
| |
| |
| |
The Maximum-Likelihood (ML) Principle | |
| |
| |
| |
Maximum Likelihood Estimation in Classical Normal Regression | |
| |
| |
| |
Consistency of Estimators | |
| |
| |
| |
Testing Linear Hypotheses | |
| |
| |
| |
Analysis of Variance | |
| |
| |
| |
Goodness of Fit | |
| |
| |
| |
Checking the Adequacy of Regression Analysis | |
| |
| |
| |
Univariate Regression | |
| |
| |
| |
Multiple Regression | |
| |
| |
| |
A Complex Example | |
| |
| |
| |
Graphical Presentation | |
| |
| |
| |
Linear Regression with Stochastic Regressors | |
| |
| |
| |
Regression and Multiple Correlation Coefficient | |
| |
| |
| |
Heterogenous Linear Estimation without Normality | |
| |
| |
| |
Heterogeneous Linear Estimation under Normality | |
| |
| |
| |
The Canonical Form | |
| |
| |
| |
Identification and Quantification of Multicollinearity | |
| |
| |
| |
Principal Components Regression | |
| |
| |
| |
Ridge Estimation | |
| |
| |
| |
Shrinkage Estimates | |
| |
| |
| |
Partial Least Squares | |
| |
| |
| |
Tests of Parameter Constancy | |
| |
| |
| |
The Chow Forecast Test | |
| |
| |
| |
The Hansen Test | |
| |
| |
| |
Tests with Recursive Estimation | |
| |
| |
| |
Test for Structural Change | |
| |
| |
| |
Total Least Squares | |
| |
| |
| |
Minimax Estimation | |
| |
| |
| |
Inequality Restrictions | |
| |
| |
| |
The Minimax Principle | |
| |
| |
| |
Censored Regression | |
| |
| |
| |
Overview | |
| |
| |
| |
LAD Estimators and Asymptotic Normality | |
| |
| |
| |
Tests of Linear Hypotheses | |
| |
| |
| |
Simultaneous Confidence Intervals | |
| |
| |
| |
Confidence Interval for the Ratio of Two Linear Parametric Functions | |
| |
| |
| |
Nonparametric Regression | |
| |
| |
| |
Estimation of the Regression Function | |
| |
| |
| |
Classification and Regression Trees (CART) | |
| |
| |
| |
Boosting and Bagging | |
| |
| |
| |
Projection Pursuit Regression | |
| |
| |
| |
Neural Networks and Nonparametric Regression | |
| |
| |
| |
Logistic Regression and Neural Networks | |
| |
| |
| |
Functional Data Analysis (FDA) | |
| |
| |
| |
Restricted Regression | |
| |
| |
| |
Problem of Selection | |
| |
| |
| |
Theory of Restricted Regression | |
| |
| |
| |
Efficiency of Selection | |
| |
| |
| |
Explicit Solution in Special Cases | |
| |
| |
| |
LINEX Loss Function | |
| |
| |
| |
Balanced Loss Function | |
| |
| |
| |
Complements | |
| |
| |
| |
Linear Models without Moments: Exercise | |
| |
| |
| |
Nonlinear Improvement of OLSE for Nonnormal Disturbances | |
| |
| |
| |
A Characterization of the Least Squares Estimator | |
| |
| |
| |
A Characterization of the Least Squares Estimator: A Lemma | |
| |
| |
| |
Exercises | |
| |
| |
| |
The Generalized Linear Regression Model | |
| |
| |
| |
Optimal Linear Estimation of � | |
| |
| |
| |
R<sub>1</sub>-Optimal Estimators | |
| |
| |
| |
R<sub>2</sub>-Optimal Estimators | |
| |
| |
| |
R<sub>3</sub>-Optimal Estimators | |
| |
| |
| |
The Aitken Estimator | |
| |
| |
| |
Misspecification of the Dispersion Matrix | |
| |
| |
| |
Heteroscedasticity and Autoregression | |
| |
| |
| |
Mixed Effects Model: Unified Theory of Linear Estimation | |
| |
| |
| |
Mixed Effects Model | |
| |
| |
| |
A Basic Lemma | |
| |
| |
| |
Estimation of X� (the Fixed Effect) | |
| |
| |
| |
Prediction of U� (the Random Effect) | |
| |
| |
| |
Estimation of ϵ | |
| |
| |
| |
Linear Mixed Models with Normal Errors and Random Effects | |
| |
| |
| |
Maximum Likelihood Estimation of Linear Mixed Models | |
| |
| |
| |
Restricted Maximum Likelihood Estimation of Linear Mixed Models | |
| |
| |
| |
Inference for Linear Mixed Models | |
| |
| |
| |
Regression-Like Equations in Econometrics | |
| |
| |
| |
Econometric Models | |
| |
| |
| |
The Reduced Form | |
| |
| |
| |
The Multivariate Regression Model | |
| |
| |
| |
The Classical Multivariate Linear Regression Model | |
| |
| |
| |
Stochastic Regression | |
| |
| |
| |
Instrumental Variable Estimator | |
| |
| |
| |
Seemingly Unrelated Regressions | |
| |
| |
| |
Measurement Error Models | |
| |
| |
| |
Simultaneous Parameter Estimation by Empirical Bayes Solutions | |
| |
| |
| |
Overview | |
| |
| |
| |
Estimation of Parameters from Different Linear Models | |
| |
| |
| |
Supplements | |
| |
| |
| |
Gauss-Markov, Aitken and Rao Least Squares Estimators | |
| |
| |
| |
Gauss-Markov Least Squares | |
| |
| |
| |
Aitken Least Squares | |
| |
| |
| |
Rao Least Squares | |
| |
| |
| |
Exercises | |
| |
| |
| |
Exact and Stochastic Linear Restrictions | |
| |
| |
| |
Use of Prior Information | |
| |
| |
| |
The Restricted Least-Squares Estimator | |
| |
| |
| |
Maximum Likelihood Estimation under Exact Restrictions | |
| |
| |
| |
Stepwise Inclusion of Exact Linear Restrictions | |
| |
| |
| |
Biased Linear Restrictions and MDE Comparison with the OLSE | |
| |
| |
| |
MDE Matrix Comparisons of Two Biased Estimators | |
| |
| |
| |
MDE Matrix Comparison of Two Linear Biased Estimators | |
| |
| |
| |
MDE Comparison of Two (Biased) Restricted Estimators | |
| |
| |
| |
Stein-Rule Estimators under Exact Restrictions | |
| |
| |
| |
Stochastic Linear Restrictions | |
| |
| |
| |
Mixed Estimator | |
| |
| |
| |
Assumptions about the Dispersion Matrix | |
| |
| |
| |
Biased Stochastic Restrictions | |
| |
| |
| |
Stein-Rule Estimators under Stochastic Restrictions | |
| |
| |
| |
Weakened Linear Restrictions | |
| |
| |
| |
Weakly (R, r)-Unbiasedness | |
| |
| |
| |
Optimal Weakly (R, r)-Unbiased Estimators | |
| |
| |
| |
Feasible Estimators-Optimal Substitution of � in <$>\hat {\beta}_1<$> (�, A) | |
| |
| |
| |
RLSE instead of the Mixed Estimator | |
| |
| |
| |
Exercises | |
| |
| |
| |
Prediction in the Generalized Regression Model | |
| |
| |
| |
Introduction | |
| |
| |
| |
Some Simple Linear Models | |
| |
| |
| |
The Constant Mean Model | |
| |
| |
| |
The Linear Trend Model | |
| |
| |
| |
Polynomial Models | |
| |
| |
| |
The Prediction Model | |
| |
| |
| |
Optimal Heterogeneous Prediction | |
| |
| |
| |
Optimal Homogeneous Prediction | |
| |
| |
| |
MDE Matrix Comparisons between Optimal and Classical Predictors | |
| |
| |
| |
Comparison of Classical and Optimal Prediction with Respect to the y* Superiority | |
| |
| |
| |
Comparison of Classical and Optimal Predictors with Respect to the X*� Superiority | |
| |
| |
| |
Prediction Regions | |
| |
| |
| |
Concepts and Definitions | |
| |
| |
| |
On q-Prediction Intervals | |
| |
| |
| |
On q-Intervals in Regression Analysis | |
| |
| |
| |
On (p, q)-Prediction Intervals | |
| |
| |
| |
Linear Utility Functions | |
| |
| |
| |
Normally Distributed Populations - Two-Sided Symmetric Intervals | |
| |
| |
| |
Onesided Infinite Intervals | |
| |
| |
| |
Utility and Length of Intervals | |
| |
| |
| |
Utility and coverage | |
| |
| |
| |
Maximal Utility and Optimal Tests | |
| |
| |
| |
Prediction Ellipsoids Based on the GLSE | |
| |
| |
| |
Comparing the Efficiency of Prediction Ellipsoids | |
| |
| |
| |
Simultaneous Prediction of Actual and Average Values of y | |
| |
| |
| |
Specification of Target Function | |
| |
| |
| |
Exact Linear Restrictions | |
| |
| |
| |
MDEP Using Ordinary Least Squares Estimator | |
| |
| |
| |
MDEP Using Restricted Estimator | |
| |
| |
| |
MDEP Matrix Comparison | |
| |
| |
| |
Stein-Rule Predictor | |
| |
| |
| |
Outside Sample Predictions | |
| |
| |
| |
Kalman Filter | |
| |
| |
| |
Dynamical and Observational Equations | |
| |
| |
| |
Some Theorems | |
| |
| |
| |
Kalman Model | |
| |
| |
| |
Exercises | |
| |
| |
| |
Sensitivity Analysis | |
| |
| |
| |
Introduction | |
| |
| |
| |
Prediction Matrix | |
| |
| |
| |
Effect of Single Observation on Estimation of Parameters | |
| |
| |
| |
Measures Based on Residuals | |
| |
| |
| |
Algebraic Consequences of Omitting an Observation | |
| |
| |
| |
Detection of Outliers | |
| |
| |
| |
Diagnostic Plots for Testing the Model Assumptions | |
| |
| |
| |
Measures Based on the Confidence Ellipsoid | |
| |
| |
| |
Partial Regression Plots | |
| |
| |
| |
Regression Diagnostics for Removing an Observation with Graphics | |
| |
| |
| |
Model Selection Criteria | |
| |
| |
| |
Akaikes Information Criterion | |
| |
| |
| |
Bayesian Information Criterion | |
| |
| |
| |
Mallows C<sub>p</sub> | |
| |
| |
| |
Example | |
| |
| |
| |
Exercises | |
| |
| |
| |
Analysis of Incomplete Data Sets | |
| |
| |
| |
Statistical Methods with Missing Data | |
| |
| |
| |
Complete Case Analysis | |
| |
| |
| |
Available Case Analysis | |
| |
| |
| |
Filling in the Missing Values | |
| |
| |
| |
Model-Based Procedures | |
| |
| |
| |
Missing-Data Mechanisms | |
| |
| |
| |
Missing Indicator Matrix | |
| |
| |
| |
Missing Completely at Random | |
| |
| |
| |
Missing at Random | |
| |
| |
| |
Nonignorable Nonresponse | |
| |
| |
| |
Missing Pattern | |
| |
| |
| |
Missing Data in the Response | |
| |
| |
| |
Least-Squares Analysis for Filled-up Data-Yates Procedure | |
| |
| |
| |
Analysis of Covariance-Bartlett's Method | |
| |
| |
| |
Shrinkage Estimation by Yates Procedure | |
| |
| |
| |
Shrinkage Estimators | |
| |
| |
| |
Efficiency Properties | |
| |
| |
| |
Missing Values in the X-Matrix | |
| |
| |
| |
General Model | |
| |
| |
| |
Missing Values and Loss in Efficiency | |
| |
| |
| |
Methods for Incomplete X-Matrices | |
| |
| |
| |
Complete Case Analysis | |
| |
| |
| |
Available Case Analysis | |
| |
| |
| |
Maximum-Likelihood Methods | |
| |
| |
| |
Imputation Methods for Incomplete X-Matrices | |
| |
| |
| |
Maximum-Likelihood Estimates of Missing Values | |
| |
| |
| |
Zero-Order Regression | |
| |
| |
| |
First-Order Regression | |
| |
| |
| |
Multiple Imputation | |
| |
| |
| |
Weighted Mixed Regression | |
| |
| |
| |
The Two-Stage WMRE | |
| |
| |
| |
Assumptions about the Missing Mechanism | |
| |
| |
| |
Regression Diagnostics to Identify Non-MCAR Processes | |
| |
| |
| |
Comparison of the Means | |
| |
| |
| |
Comparing the Variance-Covariance Matrices | |
| |
| |
| |
Diagnostic Measures from Sensitivity Analysis | |
| |
| |
| |
Distribution of the Measures and Test Procedure | |
| |
| |
| |
Treatment of Nonignorable Nonresponse | |
| |
| |
| |
Joint Distribution of (X,Y) with Missing Values Only in Y | |
| |
| |
| |
Conditional Distribution of Y Given X with Missing Values Only in Y | |
| |
| |
| |
Conditional Distribution of Y Given X with Missing Values Only in X | |
| |
| |
| |
Other Approaches | |
| |
| |
| |
Further Literature | |
| |
| |
| |
Exercises | |
| |
| |
| |
Robust Regression | |
| |
| |
| |
Overview | |
| |
| |
| |
Least Absolute Deviation Estimators - Univariate Case | |
| |
| |
| |
M-Estimates: Univariate Case | |
| |
| |
| |
Asymptotic Distributions of LAD Estimators | |
| |
| |
| |
Univariate Case | |
| |
| |
| |
Multivariate Case | |
| |
| |
| |
General M-Estimates | |
| |
| |
| |
Tests of Significance | |
| |
| |
| |
Models for Categorical Response Variables | |
| |
| |
| |
Generalized Linear Models | |
| |
| |
| |
Extension of the Regression Model | |
| |
| |
| |
Structure of the Generalized Linear Model | |
| |
| |
| |
Score Function and Information Matrix | |
| |
| |
| |
Maximum-Likelihood Estimation | |
| |
| |
| |
Testing of Hypotheses and Goodness of Fit | |
| |
| |
| |
Overdispersion | |
| |
| |
| |
Quasi Loglikelihood | |
| |
| |
| |
Contingency Tables | |
| |
| |
| |
Overview | |
| |
| |
| |
Ways of Comparing Proportions | |
| |
| |
| |
Sampling in Two-Way Contingency Tables | |
| |
| |
| |
Likelihood Function and Maximum-Likelihood Estimates | |
| |
| |
| |
Testing the Goodness of Fit | |
| |
| |
| |
GLM for Binary Response | |
| |
| |
| |
Logit Models and Logistic Regression | |
| |
| |
| |
Testing the Model | |
| |
| |
| |
Distribution Function as a Link Function | |
| |
| |
| |
Logit Models for Categorical Data | |
| |
| |
| |
Goodness of Fit-Likelihood-Ratio Test | |
| |
| |
| |
Loglinear Models for Categorical Variables | |
| |
| |
| |
Two-Way Contingency Tables | |
| |
| |
| |
Three-Way Contingency Tables | |
| |
| |
| |
The Special Case of Binary Response | |
| |
| |
| |
Coding of Categorical Explanatory Variables | |
| |
| |
| |
Dummy and Effect Coding | |
| |
| |
| |
Coding of Response Models | |
| |
| |
| |
Coding of Models for the Hazard Rate | |
| |
| |
| |
Extensions to Dependent Binary Variables | |
| |
| |
| |
Overview | |
| |
| |
| |
Modeling Approaches for Correlated Response | |
| |
| |
| |
Quasi-Likelihood Approach for Correlated Binary Response | |
| |
| |
| |
The GEE Method by Liang and Zeger | |
| |
| |
| |
Properties of the GEE Estimate <$>\hat {\beta}_G<$> | |
| |
| |
| |
Efficiency of the GEE and IEE Methods | |
| |
| |
| |
Choice of the Quasi-Correlation Matrix R<sub>t</sub>(�) | |
| |
| |
| |
Bivariate Binary Correlated Response Variables | |
| |
| |
| |
The GEE Method | |
| |
| |
| |
The IEE Method | |
| |
| |
| |
An Example from the Field of Dentistry | |
| |
| |
| |
Full Likelihood Approach for Marginal Models | |
| |
| |
| |
Exercises | |
| |
| |
| |
Matrix Algebra | |
| |
| |
| |
Overview | |
| |
| |
| |
Trace of a Matrix | |
| |
| |
| |
Determinant of a Matrix | |
| |
| |
| |
Inverse of a Matrix | |
| |
| |
| |
Orthogonal Matrices | |
| |
| |
| |
Rank of a Matrix | |
| |
| |
| |
Range and Null Space | |
| |
| |
| |
Eigenvalues and Eigenvectors | |
| |
| |
| |
Decomposition of Matrices | |
| |
| |
| |
Definite Matrices and Quadratic Forms | |
| |
| |
| |
Idempotent Matrices | |
| |
| |
| |
Generalized Inverse | |
| |
| |
| |
Projectors | |
| |
| |
| |
Functions of Normally Distributed Variables | |
| |
| |
| |
Differentiation of Scalar Functions of Matrices | |
| |
| |
| |
Miscellaneous Results, Stochastic Convergence | |
| |
| |
| |
Tables | |
| |
| |
| |
Software for Linear Regression Models | |
| |
| |
| |
Software | |
| |
| |
| |
Special-Purpose Software | |
| |
| |
| |
Resources | |
| |
| |
References | |
| |
| |
Index | |