| |
| |
| Figures and Tables | |
| |
| |
| Preface | |
| |
| |
| Acknowledgments | |
| |
| |
| The Author | |
| |
| |
| |
| Introduction | |
| |
| |
| Learning Objectives | |
| |
| |
| The Evaluation Framework | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Discussion Questions | |
| |
| |
| |
| Describing the Program | |
| |
| |
| Learning Objectives | |
| |
| |
| Motivations for Describing the Program | |
| |
| |
| Common Mistakes Evaluators Make When Describing the Program | |
| |
| |
| Conducting the Initial Informal Interviews | |
| |
| |
| Pitfalls in Describing Programs | |
| |
| |
| The Program Is Alive, and So Is Its Description | |
| |
| |
| Program Theory | |
| |
| |
| The Program Logic Model | |
| |
| |
| Challenges of Programs with Multiple Sites | |
| |
| |
| Program Implementation Model | |
| |
| |
| Program Theory and Program Logic Model Examples | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Discussion Questions | |
| |
| |
| |
| Laying the Evaluation Groundwork | |
| |
| |
| Learning Objectives | |
| |
| |
| Evaluation Approaches | |
| |
| |
| Framing Evaluation Questions | |
| |
| |
| Insincere Reasons for Evaluation | |
| |
| |
| Who Will Do the Evaluation? | |
| |
| |
| External Evaluators | |
| |
| |
| Internal Evaluators | |
| |
| |
| Confidentiality and Ownership of Evaluation Ethics | |
| |
| |
| Building a Knowledge Base from Evaluations | |
| |
| |
| High Stakes Testing | |
| |
| |
| The Evaluation Report | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Discussion Questions | |
| |
| |
| |
| Causation | |
| |
| |
| Learning Objectives | |
| |
| |
| Necessary and Sufficient | |
| |
| |
| Types of Effects | |
| |
| |
| Lagged Effects | |
| |
| |
| Permanency of Effects | |
| |
| |
| Functional Form of Impact | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Discussion Questions | |
| |
| |
| |
| The Prisms of Validity | |
| |
| |
| Learning Objectives | |
| |
| |
| Statistical Conclusion Validity | |
| |
| |
| Small Sample Sizes | |
| |
| |
| Measurement Error | |
| |
| |
| Unclear Questions | |
| |
| |
| Unreliable Treatment Implementation | |
| |
| |
| Fishing | |
| |
| |
| Internal Validity | |
| |
| |
| Threat of History | |
| |
| |
| Threat of Maturation | |
| |
| |
| Selection | |
| |
| |
| Mortality | |
| |
| |
| Testing | |
| |
| |
| Statistical Regression | |
| |
| |
| Instrumentation | |
| |
| |
| Diffusion of Treatments | |
| |
| |
| Compensatory Equalization of Treatments | |
| |
| |
| Compensatory Rivalry and Resentful Demoralization | |
| |
| |
| Construct Validity | |
| |
| |
| Mono-Operation Bias | |
| |
| |
| Mono-Method Bias | |
| |
| |
| External Validity | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Discussion Questions | |
| |
| |
| |
| Attributing Outcomes to the Program: Quasi-Experimental Design | |
| |
| |
| Learning Objectives | |
| |
| |
| Quasi-Experimental Notation | |
| |
| |
| Frequently Used Designs That Do Not Show Causation | |
| |
| |
| One-Group Posttest-Only | |
| |
| |
| Posttest-Only with Nonequivalent Groups | |
| |
| |
| Participants' Pretest-Posttest | |
| |
| |
| Designs That Generally Permit Causal Inferences | |
| |
| |
| Untreated Control Group Design with Pretest and Posttest | |
| |
| |
| Delayed Treatment Control Group | |
| |
| |
| Different Samples Design | |
| |
| |
| Nonequivalent Observations Drawn from One Group | |
| |
| |
| Nonequivalent Groups Using Switched Measures | |
| |
| |
| Cohort Designs | |
| |
| |
| Time Series Designs | |
| |
| |
| Archival Data | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Discussion Questions | |
| |
| |
| |
| Collecting Data | |
| |
| |
| Learning Objectives | |
| |
| |
| Informal Interviews | |
| |
| |
| Focus Groups | |
| |
| |
| Survey Design | |
| |
| |
| Sampling | |
| |
| |
| Ways to Collect Survey Data | |
| |
| |
| Anonymity and Confidentiality | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Discussion Questions | |
| |
| |
| |
| Conclusions | |
| |
| |
| Learning Objectives | |
| |
| |
| Using Evaluation Tools to Develop Grant Proposals | |
| |
| |
| Hiring an Evaluation Consultant | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Discussion Questions | |
| |
| |
| |
| American Community Survey | |
| |
| |
| Glossary | |
| |
| |
| References | |
| |
| |
| Index | |