| |
| |
| Preface | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Drawing Inferences from Intuitions, Anecdotes, and Correlations | |
| |
| |
| |
| Experiments as a Solution to the Problem of Unobserved Confounders | |
| |
| |
| |
| Experiments as Fair Tests | |
| |
| |
| |
| Field Experiments | |
| |
| |
| |
| Advantages and Disadvantages of Experimenting in Real-World Settings | |
| |
| |
| |
| Naturally Occurring Experiments and Quasi-Experiments | |
| |
| |
| |
| Plan of the Book | |
| |
| |
| Suggested Readings | |
| |
| |
| Exercises | |
| |
| |
| |
| Causal Inference and Experimentation | |
| |
| |
| |
| Potential Outcomes | |
| |
| |
| |
| Average Treatment Effects | |
| |
| |
| |
| Random Sampling and Expectations | |
| |
| |
| |
| Random Assignment and Unbiased Inference | |
| |
| |
| |
| The Mechanics of Random Assignment | |
| |
| |
| |
| The Threat of Selection Bias When Random Assignment Is Not Used | |
| |
| |
| |
| Two Core Assumptions about Potential Outcomes | |
| |
| |
| |
| Excludability | |
| |
| |
| |
| Non-interference | |
| |
| |
| Summary | |
| |
| |
| Suggested Readings | |
| |
| |
| Exercises | |
| |
| |
| |
| Sampling Distributions, Statistical Inference, and Hypothesis Testing | |
| |
| |
| |
| Sampling Distributions | |
| |
| |
| |
| The Standard Error as a Measure of Uncertainty | |
| |
| |
| |
| Estimating Sampling Variability 59 3.4- Hypothesis Testing | |
| |
| |
| |
| Confidence Intervals | |
| |
| |
| |
| Sampling Distributions for Experiments That Use Block or Cluster Random Assignment | |
| |
| |
| |
| Block Random Assignment | |
| |
| |
| |
| Matched Pair Design | |
| |
| |
| |
| Summary of the Advantages and Disadvantages of Blocking | |
| |
| |
| |
| Cluster Random Assignment | |
| |
| |
| Summary | |
| |
| |
| Suggested Readings | |
| |
| |
| Exercises | |
| |
| |
| |
| Power | |
| |
| |
| |
| Using Covariates in Experimental Design and Analysis | |
| |
| |
| |
| Using Covariates to Rescale Outcomes | |
| |
| |
| |
| Adjusting for Covariates Using Regression | |
| |
| |
| |
| Covariate Imbalance and the Detection of Administrative Errors | |
| |
| |
| |
| Blocked Randomization and Covariate Adjustment | |
| |
| |
| |
| Analysis of Block Randomized Experiments with Treatment Probabilities That Vary by Block | |
| |
| |
| Summary | |
| |
| |
| Suggested Readings | |
| |
| |
| Exercises | |
| |
| |
| |
| One-Sided Noncompliance | |
| |
| |
| |
| New Definitions and Assumptions | |
| |
| |
| |
| Denning Causal Effects for the Case of One-Sided Noncompliance 13 | |
| |
| |
| |
| The Non-interference Assumption for Experiments That Encounter Noncompliance | |
| |
| |
| |
| The Excludability Assumption for One-Sided Noncompliance | |
| |
| |
| |
| Average Treatment Effects, Intent-to-Treat Effects, and Complier Average Causal Effects | |
| |
| |
| |
| Identification of the CACE | |
| |
| |
| |
| Estimation | |
| |
| |
| |
| Avoiding Common Mistakes | |
| |
| |
| |
| Evaluating the Assumptions Required to Identify the CACE 15 | |
| |
| |
| |
| Non-interference Assumption | |
| |
| |
| |
| Exclusion Restriction | |
| |
| |
| |
| Statistical Inference | |
| |
| |
| |
| Designing Experiments in Anticipation of Noncompliance | |
| |
| |
| |
| Estimating Treatment Effects When Some Subjects Receive "Partial Treatment" | |
| |
| |
| Summary | |
| |
| |
| Suggested Readings | |
| |
| |
| Exercises | |
| |
| |
| |
| Two-Sided Noncompliance | |
| |
| |
| |
| Two-Sided Noncompliance: New Definitions and Assumptions | |
| |
| |
| |
| ITT, ITT<sub>D</sub>, and CACE under Two-Sided Noncompliance | |
| |
| |
| |
| A Numerical Illustration of the Role of Monotonicity | |
| |
| |
| |
| Estimation of the CACE: An Example | |
| |
| |
| |
| Discussion of Assumptions | |
| |
| |
| |
| Monotonicity | |
| |
| |
| |
| Exclusion Restriction | |
| |
| |
| |
| Random Assignment | |
| |
| |
| |
| Design Suggestions | |
| |
| |
| |
| Downstream Experimentation | |
| |
| |
| Summary | |
| |
| |
| Suggested Readings | |
| |
| |
| Exercises | |
| |
| |
| |
| Attrition | |
| |
| |
| |
| Conditions Under Which Attrition Leads to Bias | |
| |
| |
| |
| Special Forms of Attrition | |
| |
| |
| |
| Redefining the Estimand When Attrition Is Not a Function of Treatment Assignment | |
| |
| |
| |
| Placing Bounds on the Average Treatment Effect | |
| |
| |
| |
| Addressing Attrition: An Empirical Example | |
| |
| |
| |
| Addressing Attrition with Additional Data Collection | |
| |
| |
| |
| Two Frequently Asked Questions | |
| |
| |
| Summary | |
| |
| |
| Suggested Readings | |
| |
| |
| Exercises | |
| |
| |
| |
| Optimal Sample Allocation for Second-Round Sampling | |
| |
| |
| |
| Interference between Experimental Units | |
| |
| |
| |
| Identifying Causal Effects in the Presence of Localized Spillover | |
| |
| |
| |
| Spatial Spillover | |
| |
| |
| |
| Using Nonexperimental Units to Investigate Spillovers | |
| |
| |
| |
| An Example of Spatial Spillovers in Two Dimensions | |
| |
| |
| |
| Within-Subjects Design and Time-Series Experiments | |
| |
| |
| |
| Waitlist Designs (Also Known as Stepped-Wedge Designs) | |
| |
| |
| Summary | |
| |
| |
| Suggested Readings | |
| |
| |
| Exercises | |
| |
| |
| |
| Heterogeneous Treatment Effects | |
| |
| |
| |
| Limits to What Experimental Data Tell Us about Treatment Effect Heterogeneity | |
| |
| |
| |
| Bounding Var (�) and Testing for Heterogeneity | |
| |
| |
| |
| Two Approaches to the Exploration of Heterogeneity: Covariates and Design | |
| |
| |
| |
| Assessmg Treatment-by-Covariate Interactions | |
| |
| |
| |
| Caution Is Required When Interpreting Treatment-by-Covariate Interactions | |
| |
| |
| |
| Assessing Treatment-by-Treatment Interactions | |
| |
| |
| |
| Using Regression to Model Treatment Effect Heterogeneity | |
| |
| |
| |
| Automating the Search for Interactions | |
| |
| |
| Summary | |
| |
| |
| Suggested Readings | |
| |
| |
| Exercises | |
| |
| |
| |
| Mediation | |
| |
| |
| |
| Regression-Based Approaches to Mediation | |
| |
| |
| |
| Mediation Analysis from a Potential Outcomes Perspective | |
| |
| |
| |
| Why Experimental Analysis of Mediators Is Challenging | |
| |
| |
| |
| Ruling Out Mediators? | |
| |
| |
| |
| What about Experiments That Manipulate the Mediator? | |
| |
| |
| |
| Implicit Mediation Analysis | |
| |
| |
| Summary | |
| |
| |
| Suggested Readings | |
| |
| |
| Exercises | |
| |
| |
| |
| Treatment Postcards Mailed to Michigan Households | |
| |
| |
| |
| Integration of Research Findings | |
| |
| |
| |
| Estimation of Population Average Treatment Effects | |
| |
| |
| |
| A Bayesian Framework for Interpreting Research Findings | |
| |
| |
| |
| Replication and Integration of Experimental Findings: An Example | |
| |
| |
| |
| Treatments That Vary in Intensity: Extrapolation and Statistical Modeling | |
| |
| |
| Summary | |
| |
| |
| Suggested Readings | |
| |
| |
| Exercises | |
| |
| |
| |
| Instructive Examples of Experimental Design | |
| |
| |
| |
| Using Experimental Design to Distinguish between Competing Theories | |
| |
| |
| |
| Oversampling Subjects Based on Their Anticipated Response to Treatment | |
| |
| |
| |
| Comprehensive Measurement of Outcomes | |
| |
| |
| |
| Factorial Design and Special Cases of Non-interference | |
| |
| |
| |
| Design and Analysis of Experiments In Which Treatments Vary with Subjects' Characteristics | |
| |
| |
| |
| Design and Analysis of Experiments In Which Failure to Receive Treatment Has a Causal Effect | |
| |
| |
| |
| Addressing Complications Posed by Missing Data | |
| |
| |
| Summary | |
| |
| |
| Suggested Readings | |
| |
| |
| Exercises | |
| |
| |
| |
| Writing a Proposal, Research Report, and Journal Article | |
| |
| |
| |
| Writing the Proposal | |
| |
| |
| |
| Writing the Research Report | |
| |
| |
| |
| Writing the Journal Article | |
| |
| |
| |
| Archiving Data | |
| |
| |
| Summary | |
| |
| |
| Suggested Readings | |
| |
| |
| Exercises | |
| |
| |
| |
| Protection of Human Subjects | |
| |
| |
| |
| Regulatory Guidelines | |
| |
| |
| |
| Guidelines for Keeping Field Experiments within Regulatory Boundaries | |
| |
| |
| |
| Suggested Field Experiments for Class Projects | |
| |
| |
| |
| Crafting Your Own Experiment | |
| |
| |
| |
| Suggested Experimental Topics for Practicum Exercises | |
| |
| |
| References | |
| |
| |
| Index | |