| |
| |
Preface | |
| |
| |
Acknowledgements | |
| |
| |
| |
An Introduction to Survey Methodology | |
| |
| |
| |
Introduction | |
| |
| |
| |
A Brief History of Survey Research | |
| |
| |
| |
The Purposes of Surveys | |
| |
| |
| |
The Development of Standardized Questioning | |
| |
| |
| |
The Development of Sampling Methods | |
| |
| |
| |
The Development of Data Collection Methods | |
| |
| |
| |
Some Examples of Ongoing Surveys | |
| |
| |
| |
The National Crime Victimization Survey | |
| |
| |
| |
The National Survey on Drug Use and Health | |
| |
| |
| |
The Survey of Consumers | |
| |
| |
| |
The National Assessment of Educational Progress | |
| |
| |
| |
The Behavioral Risk Factor Surveillance System | |
| |
| |
| |
The Current Employment Statistics Program | |
| |
| |
| |
What Can We Learn From the Six Example Surveys? | |
| |
| |
| |
What is Survey Methodology? | |
| |
| |
| |
The Challenge of Survey Methodology | |
| |
| |
| |
About this Book | |
| |
| |
| |
Inference and Error in Surveys | |
| |
| |
| |
Introduction | |
| |
| |
| |
The Lifecycle of a Survey From a Design Perspective | |
| |
| |
| |
Constructs | |
| |
| |
| |
Measurement | |
| |
| |
| |
Response | |
| |
| |
| |
Edited Response | |
| |
| |
| |
The Target Population | |
| |
| |
| |
The Frame Population | |
| |
| |
| |
The Sample | |
| |
| |
| |
The Respondents | |
| |
| |
| |
Postsurvey Adjustments | |
| |
| |
| |
How Design Becomes Process | |
| |
| |
| |
The Lifecycle of a Survey from A Quality Perspective | |
| |
| |
| |
The Observational Gap between Constructs and Measures | |
| |
| |
| |
Measurement Error: The Observational Gap between the Ideal Measurement and the Response Obtained | |
| |
| |
| |
Processing Error: the Observational Gap between the Variable Used in Estimation and that Provided by the Respondent | |
| |
| |
| |
Coverage Error: the Nonobservational Gap between the Target Population and the Sampling Frame | |
| |
| |
| |
Sampling Error: The Nonobservational Gap between the Sample and the Sampling Frame | |
| |
| |
| |
Nonresponse Error: The Nonobservational Gap between the Sampling and the Respondent Pool | |
| |
| |
| |
Adjustment Error | |
| |
| |
| |
Putting It All Together | |
| |
| |
| |
Error Notions in Different Kinds of Statistics | |
| |
| |
| |
Summary | |
| |
| |
| |
Target Populations, Sampling Frames, and Coverage Error | |
| |
| |
| |
Introduction | |
| |
| |
| |
Populations and Frames | |
| |
| |
| |
Coverage Properties of Sampling Frames | |
| |
| |
| |
Undercoverage | |
| |
| |
| |
Ineligible Units | |
| |
| |
| |
Clustering of Target Population Elements Within Frame Elements | |
| |
| |
| |
Duplication of Target Population Elements in Sampling Frames | |
| |
| |
| |
Complicated Mappings between Frame and Target Population Elements | |
| |
| |
| |
Common Target Populations and Their Frame Issues | |
| |
| |
| |
Households and Persons | |
| |
| |
| |
Customers, Employees, or Members of an Organization | |
| |
| |
| |
Organizations | |
| |
| |
| |
Events | |
| |
| |
| |
Rare Populations | |
| |
| |
| |
Coverage Error | |
| |
| |
| |
Reducing Undercoverage | |
| |
| |
| |
The Half-Open Interval | |
| |
| |
| |
Multiplicity Sampling | |
| |
| |
| |
Multiple Frame Designs | |
| |
| |
| |
Increasing Coverage While Including More Ineligible Elements | |
| |
| |
| |
Summary | |
| |
| |
| |
Sample Design and Sampling Error | |
| |
| |
| |
Introduction | |
| |
| |
| |
Samples and Estimates | |
| |
| |
| |
Simple Random Sampling | |
| |
| |
| |
Cluster Sampling | |
| |
| |
| |
The Design Effect and Within-Cluster Homogeneity | |
| |
| |
| |
Subsampling within Selected Clusters | |
| |
| |
| |
Stratification and Stratified Sampling | |
| |
| |
| |
Proportionate Allocation to Strata | |
| |
| |
| |
Disproportionate Allocation to Strata | |
| |
| |
| |
Systematic Selection | |
| |
| |
| |
Complications in Practice | |
| |
| |
| |
Two-Stage Cluster Designs with Probabilities Proportionate to Size (PPS) | |
| |
| |
| |
Multistage and Other Complex Designs | |
| |
| |
| |
How Complex Sample Designs are Described: The Sample Design for the NCVS | |
| |
| |
| |
Sampling Persons in US Telephone Households | |
| |
| |
| |
Summary | |
| |
| |
| |
Methods of Data Collection | |
| |
| |
| |
Alternative Methods of Data Collection | |
| |
| |
| |
Degree of Interviewer Involvement | |
| |
| |
| |
Degree of Interaction with the Respondent | |
| |
| |
| |
Degree of Privacy | |
| |
| |
| |
Channels of Communication | |
| |
| |
| |
Technology Use | |
| |
| |
| |
Implications of these Dimensions | |
| |
| |
| |
Choosing the Appropriate Method | |
| |
| |
| |
Effects of Different Data Collection Methods on Survey Errors | |
| |
| |
| |
Measuring the Marginal Effect of Mode | |
| |
| |
| |
Sampling Frame and Sample Design Implications of Mode Selection | |
| |
| |
| |
Coverage Implications of Mode Selection | |
| |
| |
| |
Nonresponse Implications of Mode Selection | |
| |
| |
| |
Measurement Quality Implications of Mode Selection | |
| |
| |
| |
Cost Implications | |
| |
| |
| |
Summary on the Choice of Method | |
| |
| |
| |
Using Multiple Modes of Data Collection | |
| |
| |
| |
Summary | |
| |
| |
| |
Nonresponse in Sample Surveys | |
| |
| |
| |
Introduction | |
| |
| |
| |
Three Major Types of Unit Nonresponse | |
| |
| |
| |
Unit Nonresponse Due to Failure to Deliver the Survey Request | |
| |
| |
| |
Unit Nonresponse Due to Refusals | |
| |
| |
| |
Unit Nonresponse Due to the Inability to Provide the Requested Data | |
| |
| |
| |
How Does Nonresponse Affect the Quality of Survey Statistics? | |
| |
| |
| |
Computing Response and Nonresponse Rates | |
| |
| |
| |
Trends in Response Rates over Time | |
| |
| |
| |
Item Nonresponse | |
| |
| |
| |
Design Features to Reduce Unit Nonresponse | |
| |
| |
| |
Summary | |
| |
| |
| |
Questions and Answers in Surveys | |
| |
| |
| |
Alternatives Methods of Survey Measurement | |
| |
| |
| |
Cognitive Processes in Answering Questions | |
| |
| |
| |
Comprehension | |
| |
| |
| |
Retrieval | |
| |
| |
| |
Estimation and Judgment | |
| |
| |
| |
Reporting | |
| |
| |
| |
Other Models of the Response Process | |
| |
| |
| |
Problems in Answering Survey Questions | |
| |
| |
| |
Encoding Problems | |
| |
| |
| |
Misinterpreting the Questions | |
| |
| |
| |
Forgetting and Other Memory Problems | |
| |
| |
| |
Estimation Processes for Behavioral Questions | |
| |
| |
| |
Judgment Processes for Attitude Questions | |
| |
| |
| |
Formatting the Answer | |
| |
| |
| |
Motivated Misreporting | |
| |
| |
| |
Navigational Errors | |
| |
| |
| |
Guidelines for Writing Good Questions | |
| |
| |
| |
Nonsensitive Questions About Behavior | |
| |
| |
| |
Sensitive Questions About Behavior | |
| |
| |
| |
Attitude Questions | |
| |
| |
| |
Self-Administered Questions | |
| |
| |
| |
Summary | |
| |
| |
| |
Evaluating Survey Questions | |
| |
| |
| |
Introduction | |
| |
| |
| |
Expert Reviews | |
| |
| |
| |
Focus Groups | |
| |
| |
| |
Cognitive Interviews | |
| |
| |
| |
Field Pretests and Behavior Coding | |
| |
| |
| |
Randomized or Split-Ballot Experiments | |
| |
| |
| |
Applying Question Standards | |
| |
| |
| |
Summary of Question Evaluation Tools | |
| |
| |
| |
Linking Concepts of Measurement Quality to Statistical Estimates | |
| |
| |
| |
Validity | |
| |
| |
| |
Response Bias | |
| |
| |
| |
Reliability and Simple Response Variance | |
| |
| |
| |
Summary | |
| |
| |
| |
Survey Interviewing | |
| |
| |
| |
The Role of the Interviewer | |
| |
| |
| |
Interviewer Bias | |
| |
| |
| |
Systematic Interviewer Effects on Reporting of Socially Undesirable Attributes | |
| |
| |
| |
Systematic Interviewer Effects on Topics Related to Observable Interviewer Traits | |
| |
| |
| |
Systematic Interviewer Effects Associated with Interviewer Experience | |
| |
| |
| |
Interviewer Variance | |
| |
| |
| |
Randomization Requirements for Estimating Interviewer Variance | |
| |
| |
| |
Estimation of Interviewer Variance | |
| |
| |
| |
Strategies for Reducing Interviewer Bias | |
| |
| |
| |
The Role of the Interviewer in Motivating Respondent Behavior | |
| |
| |
| |
Changing Interviewer Behavior | |
| |
| |
| |
Strategies for Reducing Interviewer-Related Variance | |
| |
| |
| |
Minimizing Questions that Require Nonstandard Interviewer Behavior | |
| |
| |
| |
Professional, Task-Oriented Interviewer Behavior | |
| |
| |
| |
Interviewers Reading Questions as They Are Worded | |
| |
| |
| |
Interviewers Explaining the Survey Process to the Respondent | |
| |
| |
| |
Interviewers Probing Nondirectively | |
| |
| |
| |
Interviewers Recording Answers Exactly as Given | |
| |
| |
| |
Summary on Strategies to Reduce Interviewer Variance | |
| |
| |
| |
The Controversy About Standardized Interviewing | |
| |
| |
| |
Interviewer Management | |
| |
| |
| |
Interviewer Selection | |
| |
| |
| |
Interviewer Training | |
| |
| |
| |
Interviewer Supervision and Monitoring | |
| |
| |
| |
The Size of Interviewer Workloads | |
| |
| |
| |
Interviewers and Computer Use | |
| |
| |
| |
Summary | |
| |
| |
| |
Postcollection Processing of Survey Data | |
| |
| |
| |
Introduction | |
| |
| |
| |
Coding | |
| |
| |
| |
Practical Issues of Coding | |
| |
| |
| |
Theoretical Issues in Coding Activities | |
| |
| |
| |
"Field Coding"--An Intermediate Design | |
| |
| |
| |
Standard Classification Systems | |
| |
| |
| |
Other Common Coding Systems | |
| |
| |
| |
Quality Indicators in Coding | |
| |
| |
| |
Summary of Coding | |
| |
| |
| |
Entering Numeric Data into Files | |
| |
| |
| |
Editing | |
| |
| |
| |
Weighting | |
| |
| |
| |
Weighting with a First-Stage Ratio Adjustment | |
| |
| |
| |
Weighting for Differential Selection Probabilities | |
| |
| |
| |
Weighting to Adjust for Unit Nonresponse | |
| |
| |
| |
Poststratification Weighting | |
| |
| |
| |
Putting All the Weights Together | |
| |
| |
| |
Imputation for Item-missing data | |
| |
| |
| |
Sampling Variance Estimation for Complex Samples | |
| |
| |
| |
Survey Data Documentation and Metadata | |
| |
| |
| |
Summary of Postsurvey Processing | |
| |
| |
| |
Principles and Practices Related to Scientific Integrity | |
| |
| |
| |
Introduction | |
| |
| |
| |
Standards for the Conduct of Research | |
| |
| |
| |
Standards for Dealing with Clients | |
| |
| |
| |
Standards for Dealing with the Public | |
| |
| |
| |
Standards for Dealing with Respondents | |
| |
| |
| |
Legal Obligations to Survey Respondents | |
| |
| |
| |
Ethical Obligations to Respondents | |
| |
| |
| |
Informed Consent: Respect for Persons | |
| |
| |
| |
Beneficence: Protecting Respondents from Harm | |
| |
| |
| |
Efforts at Persuasion | |
| |
| |
| |
Emerging Ethical Issues | |
| |
| |
| |
Practices in Research Administration Regarding Human Subject Issues | |
| |
| |
| |
Research About Informed Consent Protocols in Surveys | |
| |
| |
| |
Research on Respondents' Reactions to the Content of Informed Consent Protocols | |
| |
| |
| |
Research on Informed Consent Complications in Survey Methodological Studies | |
| |
| |
| |
Research on Written versus Oral Informed Consent | |
| |
| |
| |
Summary of Research on Informed Consent in Surveys | |
| |
| |
| |
Procedures to Detect and Repair Interviewer Falsification | |
| |
| |
| |
Disclosure Limitation Procedures | |
| |
| |
| |
Administrative Procedures to Limit Identifiability of Survey Materials | |
| |
| |
| |
Restricting Access to the Data Only to Those Subscribing to a Pledge of Confidentiality | |
| |
| |
| |
Restricting the Contents of the Survey Data That May Be Released | |
| |
| |
| |
Summary and Conclusions | |
| |
| |
| |
FAQs about Survey Methodology | |
| |
| |
| |
Introduction | |
| |
| |
| |
The Questions and Their Answers | |
| |
| |
References | |
| |
| |
Index | |