| |

| |

Preface to the First Edition | |

| |

| |

Preface to the Second Edition | |

| |

| |

Preface to the Third Edition | |

| |

| |

| |

Introduction | |

| |

| |

| |

Linear Models and Regression Analysis | |

| |

| |

| |

Plan of the Book | |

| |

| |

| |

The Simple Linear Regression Model | |

| |

| |

| |

The Linear Model | |

| |

| |

| |

Least Squares Estimation | |

| |

| |

| |

Direct Regression Method | |

| |

| |

| |

Properties of the Direct Regression Estimators | |

| |

| |

| |

Centered Model | |

| |

| |

| |

No Intercept Term Model | |

| |

| |

| |

Maximum Likelihood Estimation | |

| |

| |

| |

Testing of Hypotheses and Confidence Interval Estimation | |

| |

| |

| |

Analysis of Variance | |

| |

| |

| |

Goodness of Fit of Regression | |

| |

| |

| |

Reverse Regression Method | |

| |

| |

| |

Orthogonal Regression Method | |

| |

| |

| |

Reduced Major Axis Regression Method | |

| |

| |

| |

Least Absolute Deviation Regression Method | |

| |

| |

| |

Estimation of Parameters when X Is Stochastic | |

| |

| |

| |

The Multiple Linear Regression Model and Its Extensions | |

| |

| |

| |

The Linear Model | |

| |

| |

| |

The Principle of Ordinary Least Squares (OLS) | |

| |

| |

| |

Geometric Properties of OLS | |

| |

| |

| |

Best Linear Unbiased Estimation | |

| |

| |

| |

Basic Theorems | |

| |

| |

| |

Linear Estimators | |

| |

| |

| |

Mean Dispersion Error | |

| |

| |

| |

Estimation (Prediction) of the Error Term ï¿½ and ï¿½<sup>2</sup> | |

| |

| |

| |

Classical Regression under Normal Errors | |

| |

| |

| |

The Maximum-Likelihood (ML) Principle | |

| |

| |

| |

Maximum Likelihood Estimation in Classical Normal Regression | |

| |

| |

| |

Consistency of Estimators | |

| |

| |

| |

Testing Linear Hypotheses | |

| |

| |

| |

Analysis of Variance | |

| |

| |

| |

Goodness of Fit | |

| |

| |

| |

Checking the Adequacy of Regression Analysis | |

| |

| |

| |

Univariate Regression | |

| |

| |

| |

Multiple Regression | |

| |

| |

| |

A Complex Example | |

| |

| |

| |

Graphical Presentation | |

| |

| |

| |

Linear Regression with Stochastic Regressors | |

| |

| |

| |

Regression and Multiple Correlation Coefficient | |

| |

| |

| |

Heterogenous Linear Estimation without Normality | |

| |

| |

| |

Heterogeneous Linear Estimation under Normality | |

| |

| |

| |

The Canonical Form | |

| |

| |

| |

Identification and Quantification of Multicollinearity | |

| |

| |

| |

Principal Components Regression | |

| |

| |

| |

Ridge Estimation | |

| |

| |

| |

Shrinkage Estimates | |

| |

| |

| |

Partial Least Squares | |

| |

| |

| |

Tests of Parameter Constancy | |

| |

| |

| |

The Chow Forecast Test | |

| |

| |

| |

The Hansen Test | |

| |

| |

| |

Tests with Recursive Estimation | |

| |

| |

| |

Test for Structural Change | |

| |

| |

| |

Total Least Squares | |

| |

| |

| |

Minimax Estimation | |

| |

| |

| |

Inequality Restrictions | |

| |

| |

| |

The Minimax Principle | |

| |

| |

| |

Censored Regression | |

| |

| |

| |

Overview | |

| |

| |

| |

LAD Estimators and Asymptotic Normality | |

| |

| |

| |

Tests of Linear Hypotheses | |

| |

| |

| |

Simultaneous Confidence Intervals | |

| |

| |

| |

Confidence Interval for the Ratio of Two Linear Parametric Functions | |

| |

| |

| |

Nonparametric Regression | |

| |

| |

| |

Estimation of the Regression Function | |

| |

| |

| |

Classification and Regression Trees (CART) | |

| |

| |

| |

Boosting and Bagging | |

| |

| |

| |

Projection Pursuit Regression | |

| |

| |

| |

Neural Networks and Nonparametric Regression | |

| |

| |

| |

Logistic Regression and Neural Networks | |

| |

| |

| |

Functional Data Analysis (FDA) | |

| |

| |

| |

Restricted Regression | |

| |

| |

| |

Problem of Selection | |

| |

| |

| |

Theory of Restricted Regression | |

| |

| |

| |

Efficiency of Selection | |

| |

| |

| |

Explicit Solution in Special Cases | |

| |

| |

| |

LINEX Loss Function | |

| |

| |

| |

Balanced Loss Function | |

| |

| |

| |

Complements | |

| |

| |

| |

Linear Models without Moments: Exercise | |

| |

| |

| |

Nonlinear Improvement of OLSE for Nonnormal Disturbances | |

| |

| |

| |

A Characterization of the Least Squares Estimator | |

| |

| |

| |

A Characterization of the Least Squares Estimator: A Lemma | |

| |

| |

| |

Exercises | |

| |

| |

| |

The Generalized Linear Regression Model | |

| |

| |

| |

Optimal Linear Estimation of ï¿½ | |

| |

| |

| |

R<sub>1</sub>-Optimal Estimators | |

| |

| |

| |

R<sub>2</sub>-Optimal Estimators | |

| |

| |

| |

R<sub>3</sub>-Optimal Estimators | |

| |

| |

| |

The Aitken Estimator | |

| |

| |

| |

Misspecification of the Dispersion Matrix | |

| |

| |

| |

Heteroscedasticity and Autoregression | |

| |

| |

| |

Mixed Effects Model: Unified Theory of Linear Estimation | |

| |

| |

| |

Mixed Effects Model | |

| |

| |

| |

A Basic Lemma | |

| |

| |

| |

Estimation of Xï¿½ (the Fixed Effect) | |

| |

| |

| |

Prediction of Uï¿½ (the Random Effect) | |

| |

| |

| |

Estimation of ϵ | |

| |

| |

| |

Linear Mixed Models with Normal Errors and Random Effects | |

| |

| |

| |

Maximum Likelihood Estimation of Linear Mixed Models | |

| |

| |

| |

Restricted Maximum Likelihood Estimation of Linear Mixed Models | |

| |

| |

| |

Inference for Linear Mixed Models | |

| |

| |

| |

Regression-Like Equations in Econometrics | |

| |

| |

| |

Econometric Models | |

| |

| |

| |

The Reduced Form | |

| |

| |

| |

The Multivariate Regression Model | |

| |

| |

| |

The Classical Multivariate Linear Regression Model | |

| |

| |

| |

Stochastic Regression | |

| |

| |

| |

Instrumental Variable Estimator | |

| |

| |

| |

Seemingly Unrelated Regressions | |

| |

| |

| |

Measurement Error Models | |

| |

| |

| |

Simultaneous Parameter Estimation by Empirical Bayes Solutions | |

| |

| |

| |

Overview | |

| |

| |

| |

Estimation of Parameters from Different Linear Models | |

| |

| |

| |

Supplements | |

| |

| |

| |

Gauss-Markov, Aitken and Rao Least Squares Estimators | |

| |

| |

| |

Gauss-Markov Least Squares | |

| |

| |

| |

Aitken Least Squares | |

| |

| |

| |

Rao Least Squares | |

| |

| |

| |

Exercises | |

| |

| |

| |

Exact and Stochastic Linear Restrictions | |

| |

| |

| |

Use of Prior Information | |

| |

| |

| |

The Restricted Least-Squares Estimator | |

| |

| |

| |

Maximum Likelihood Estimation under Exact Restrictions | |

| |

| |

| |

Stepwise Inclusion of Exact Linear Restrictions | |

| |

| |

| |

Biased Linear Restrictions and MDE Comparison with the OLSE | |

| |

| |

| |

MDE Matrix Comparisons of Two Biased Estimators | |

| |

| |

| |

MDE Matrix Comparison of Two Linear Biased Estimators | |

| |

| |

| |

MDE Comparison of Two (Biased) Restricted Estimators | |

| |

| |

| |

Stein-Rule Estimators under Exact Restrictions | |

| |

| |

| |

Stochastic Linear Restrictions | |

| |

| |

| |

Mixed Estimator | |

| |

| |

| |

Assumptions about the Dispersion Matrix | |

| |

| |

| |

Biased Stochastic Restrictions | |

| |

| |

| |

Stein-Rule Estimators under Stochastic Restrictions | |

| |

| |

| |

Weakened Linear Restrictions | |

| |

| |

| |

Weakly (R, r)-Unbiasedness | |

| |

| |

| |

Optimal Weakly (R, r)-Unbiased Estimators | |

| |

| |

| |

Feasible Estimators-Optimal Substitution of ï¿½ in <$>\hat {\beta}_1<$> (ï¿½, A) | |

| |

| |

| |

RLSE instead of the Mixed Estimator | |

| |

| |

| |

Exercises | |

| |

| |

| |

Prediction in the Generalized Regression Model | |

| |

| |

| |

Introduction | |

| |

| |

| |

Some Simple Linear Models | |

| |

| |

| |

The Constant Mean Model | |

| |

| |

| |

The Linear Trend Model | |

| |

| |

| |

Polynomial Models | |

| |

| |

| |

The Prediction Model | |

| |

| |

| |

Optimal Heterogeneous Prediction | |

| |

| |

| |

Optimal Homogeneous Prediction | |

| |

| |

| |

MDE Matrix Comparisons between Optimal and Classical Predictors | |

| |

| |

| |

Comparison of Classical and Optimal Prediction with Respect to the y* Superiority | |

| |

| |

| |

Comparison of Classical and Optimal Predictors with Respect to the X*ï¿½ Superiority | |

| |

| |

| |

Prediction Regions | |

| |

| |

| |

Concepts and Definitions | |

| |

| |

| |

On q-Prediction Intervals | |

| |

| |

| |

On q-Intervals in Regression Analysis | |

| |

| |

| |

On (p, q)-Prediction Intervals | |

| |

| |

| |

Linear Utility Functions | |

| |

| |

| |

Normally Distributed Populations - Two-Sided Symmetric Intervals | |

| |

| |

| |

Onesided Infinite Intervals | |

| |

| |

| |

Utility and Length of Intervals | |

| |

| |

| |

Utility and coverage | |

| |

| |

| |

Maximal Utility and Optimal Tests | |

| |

| |

| |

Prediction Ellipsoids Based on the GLSE | |

| |

| |

| |

Comparing the Efficiency of Prediction Ellipsoids | |

| |

| |

| |

Simultaneous Prediction of Actual and Average Values of y | |

| |

| |

| |

Specification of Target Function | |

| |

| |

| |

Exact Linear Restrictions | |

| |

| |

| |

MDEP Using Ordinary Least Squares Estimator | |

| |

| |

| |

MDEP Using Restricted Estimator | |

| |

| |

| |

MDEP Matrix Comparison | |

| |

| |

| |

Stein-Rule Predictor | |

| |

| |

| |

Outside Sample Predictions | |

| |

| |

| |

Kalman Filter | |

| |

| |

| |

Dynamical and Observational Equations | |

| |

| |

| |

Some Theorems | |

| |

| |

| |

Kalman Model | |

| |

| |

| |

Exercises | |

| |

| |

| |

Sensitivity Analysis | |

| |

| |

| |

Introduction | |

| |

| |

| |

Prediction Matrix | |

| |

| |

| |

Effect of Single Observation on Estimation of Parameters | |

| |

| |

| |

Measures Based on Residuals | |

| |

| |

| |

Algebraic Consequences of Omitting an Observation | |

| |

| |

| |

Detection of Outliers | |

| |

| |

| |

Diagnostic Plots for Testing the Model Assumptions | |

| |

| |

| |

Measures Based on the Confidence Ellipsoid | |

| |

| |

| |

Partial Regression Plots | |

| |

| |

| |

Regression Diagnostics for Removing an Observation with Graphics | |

| |

| |

| |

Model Selection Criteria | |

| |

| |

| |

Akaikes Information Criterion | |

| |

| |

| |

Bayesian Information Criterion | |

| |

| |

| |

Mallows C<sub>p</sub> | |

| |

| |

| |

Example | |

| |

| |

| |

Exercises | |

| |

| |

| |

Analysis of Incomplete Data Sets | |

| |

| |

| |

Statistical Methods with Missing Data | |

| |

| |

| |

Complete Case Analysis | |

| |

| |

| |

Available Case Analysis | |

| |

| |

| |

Filling in the Missing Values | |

| |

| |

| |

Model-Based Procedures | |

| |

| |

| |

Missing-Data Mechanisms | |

| |

| |

| |

Missing Indicator Matrix | |

| |

| |

| |

Missing Completely at Random | |

| |

| |

| |

Missing at Random | |

| |

| |

| |

Nonignorable Nonresponse | |

| |

| |

| |

Missing Pattern | |

| |

| |

| |

Missing Data in the Response | |

| |

| |

| |

Least-Squares Analysis for Filled-up Data-Yates Procedure | |

| |

| |

| |

Analysis of Covariance-Bartlett's Method | |

| |

| |

| |

Shrinkage Estimation by Yates Procedure | |

| |

| |

| |

Shrinkage Estimators | |

| |

| |

| |

Efficiency Properties | |

| |

| |

| |

Missing Values in the X-Matrix | |

| |

| |

| |

General Model | |

| |

| |

| |

Missing Values and Loss in Efficiency | |

| |

| |

| |

Methods for Incomplete X-Matrices | |

| |

| |

| |

Complete Case Analysis | |

| |

| |

| |

Available Case Analysis | |

| |

| |

| |

Maximum-Likelihood Methods | |

| |

| |

| |

Imputation Methods for Incomplete X-Matrices | |

| |

| |

| |

Maximum-Likelihood Estimates of Missing Values | |

| |

| |

| |

Zero-Order Regression | |

| |

| |

| |

First-Order Regression | |

| |

| |

| |

Multiple Imputation | |

| |

| |

| |

Weighted Mixed Regression | |

| |

| |

| |

The Two-Stage WMRE | |

| |

| |

| |

Assumptions about the Missing Mechanism | |

| |

| |

| |

Regression Diagnostics to Identify Non-MCAR Processes | |

| |

| |

| |

Comparison of the Means | |

| |

| |

| |

Comparing the Variance-Covariance Matrices | |

| |

| |

| |

Diagnostic Measures from Sensitivity Analysis | |

| |

| |

| |

Distribution of the Measures and Test Procedure | |

| |

| |

| |

Treatment of Nonignorable Nonresponse | |

| |

| |

| |

Joint Distribution of (X,Y) with Missing Values Only in Y | |

| |

| |

| |

Conditional Distribution of Y Given X with Missing Values Only in Y | |

| |

| |

| |

Conditional Distribution of Y Given X with Missing Values Only in X | |

| |

| |

| |

Other Approaches | |

| |

| |

| |

Further Literature | |

| |

| |

| |

Exercises | |

| |

| |

| |

Robust Regression | |

| |

| |

| |

Overview | |

| |

| |

| |

Least Absolute Deviation Estimators - Univariate Case | |

| |

| |

| |

M-Estimates: Univariate Case | |

| |

| |

| |

Asymptotic Distributions of LAD Estimators | |

| |

| |

| |

Univariate Case | |

| |

| |

| |

Multivariate Case | |

| |

| |

| |

General M-Estimates | |

| |

| |

| |

Tests of Significance | |

| |

| |

| |

Models for Categorical Response Variables | |

| |

| |

| |

Generalized Linear Models | |

| |

| |

| |

Extension of the Regression Model | |

| |

| |

| |

Structure of the Generalized Linear Model | |

| |

| |

| |

Score Function and Information Matrix | |

| |

| |

| |

Maximum-Likelihood Estimation | |

| |

| |

| |

Testing of Hypotheses and Goodness of Fit | |

| |

| |

| |

Overdispersion | |

| |

| |

| |

Quasi Loglikelihood | |

| |

| |

| |

Contingency Tables | |

| |

| |

| |

Overview | |

| |

| |

| |

Ways of Comparing Proportions | |

| |

| |

| |

Sampling in Two-Way Contingency Tables | |

| |

| |

| |

Likelihood Function and Maximum-Likelihood Estimates | |

| |

| |

| |

Testing the Goodness of Fit | |

| |

| |

| |

GLM for Binary Response | |

| |

| |

| |

Logit Models and Logistic Regression | |

| |

| |

| |

Testing the Model | |

| |

| |

| |

Distribution Function as a Link Function | |

| |

| |

| |

Logit Models for Categorical Data | |

| |

| |

| |

Goodness of Fit-Likelihood-Ratio Test | |

| |

| |

| |

Loglinear Models for Categorical Variables | |

| |

| |

| |

Two-Way Contingency Tables | |

| |

| |

| |

Three-Way Contingency Tables | |

| |

| |

| |

The Special Case of Binary Response | |

| |

| |

| |

Coding of Categorical Explanatory Variables | |

| |

| |

| |

Dummy and Effect Coding | |

| |

| |

| |

Coding of Response Models | |

| |

| |

| |

Coding of Models for the Hazard Rate | |

| |

| |

| |

Extensions to Dependent Binary Variables | |

| |

| |

| |

Overview | |

| |

| |

| |

Modeling Approaches for Correlated Response | |

| |

| |

| |

Quasi-Likelihood Approach for Correlated Binary Response | |

| |

| |

| |

The GEE Method by Liang and Zeger | |

| |

| |

| |

Properties of the GEE Estimate <$>\hat {\beta}_G<$> | |

| |

| |

| |

Efficiency of the GEE and IEE Methods | |

| |

| |

| |

Choice of the Quasi-Correlation Matrix R<sub>t</sub>(ï¿½) | |

| |

| |

| |

Bivariate Binary Correlated Response Variables | |

| |

| |

| |

The GEE Method | |

| |

| |

| |

The IEE Method | |

| |

| |

| |

An Example from the Field of Dentistry | |

| |

| |

| |

Full Likelihood Approach for Marginal Models | |

| |

| |

| |

Exercises | |

| |

| |

| |

Matrix Algebra | |

| |

| |

| |

Overview | |

| |

| |

| |

Trace of a Matrix | |

| |

| |

| |

Determinant of a Matrix | |

| |

| |

| |

Inverse of a Matrix | |

| |

| |

| |

Orthogonal Matrices | |

| |

| |

| |

Rank of a Matrix | |

| |

| |

| |

Range and Null Space | |

| |

| |

| |

Eigenvalues and Eigenvectors | |

| |

| |

| |

Decomposition of Matrices | |

| |

| |

| |

Definite Matrices and Quadratic Forms | |

| |

| |

| |

Idempotent Matrices | |

| |

| |

| |

Generalized Inverse | |

| |

| |

| |

Projectors | |

| |

| |

| |

Functions of Normally Distributed Variables | |

| |

| |

| |

Differentiation of Scalar Functions of Matrices | |

| |

| |

| |

Miscellaneous Results, Stochastic Convergence | |

| |

| |

| |

Tables | |

| |

| |

| |

Software for Linear Regression Models | |

| |

| |

| |

Software | |

| |

| |

| |

Special-Purpose Software | |

| |

| |

| |

Resources | |

| |

| |

References | |

| |

| |

Index | |