| |
| |
Preface | |
| |
| |
About the Author | |
| |
| |
| |
Conducting Surveys: Everyone Is Doing It | |
| |
| |
Overview | |
| |
| |
What Is a Survey? | |
| |
| |
Examples: Surveys to Meet Policy or Program Needs | |
| |
| |
Examples: Surveys in Evaluations of Programs | |
| |
| |
Examples: Surveys for Research | |
| |
| |
When Is a Survey Best? | |
| |
| |
Examples: From an Overly Ambitious Self-Administered Questionnaire | |
| |
| |
Examples: Surveys Combined with Other Information Sources | |
| |
| |
Self-Administered Questionnaires and Interviews: The Heart of the Matter | |
| |
| |
Questions and Responses | |
| |
| |
Example: Forced-Choice Question | |
| |
| |
Example: Open-Ended Question | |
| |
| |
Survey Sample and Design | |
| |
| |
Planning for Data Analysis | |
| |
| |
Pilot Testing | |
| |
| |
Response Rate | |
| |
| |
Reporting Results | |
| |
| |
Example 1: The Look of Survey Results | |
| |
| |
Example 2: The Look of Survey Results | |
| |
| |
Example 3: The Look of Survey Results | |
| |
| |
The Friendly Competition | |
| |
| |
Reliability and Validity | |
| |
| |
Usefulness or Credibility of Results | |
| |
| |
Costs | |
| |
| |
The Special Case of Online Surveys | |
| |
| |
Checklist for Deciding the Best Uses for Online Surveys | |
| |
| |
Guidelines for Surveyors Who Work with Commercial Survey Companies | |
| |
| |
The Special Case of Cell or Mobile Phones | |
| |
| |
Making the Decision | |
| |
| |
A Survey Continuum: From Specific to General Use | |
| |
| |
Example: Survey with a Specific Use | |
| |
| |
Example: Survey with a General Use | |
| |
| |
Ethics, Privacy, and Confidentiality | |
| |
| |
Informed Consent | |
| |
| |
Contents of an Informed-Consent Form | |
| |
| |
The Internet and Ethical Surveys | |
| |
| |
Example Questionnaire: Maintaining an Ethically Sound Online Survey | |
| |
| |
Example: Informed-Consent Form for an Online. Survey | |
| |
| |
Summing Up | |
| |
| |
Think About This | |
| |
| |
Articles | |
| |
| |
| |
The Survey Form: Questions, Scales, and Appearance | |
| |
| |
Overview | |
| |
| |
The Content Is the Message | |
| |
| |
Define the Terms | |
| |
| |
Select Your Information Needs or Hypotheses | |
| |
| |
Make Sure You Can Get the Information | |
| |
| |
Do Not Ask for Information Unless You Can Act on It | |
| |
| |
Example: Plan for Survey of Satisfaction with the Youth Center | |
| |
| |
Writing Questions | |
| |
| |
Open-Ended and Closed Questions | |
| |
| |
Example: Open-Ended Question | |
| |
| |
Example: Closed Question | |
| |
| |
Example: Open-Ended Question for Elementary School Teaching Program | |
| |
| |
Example: Closed Question for Elementary School Teaching Program | |
| |
| |
Making the Decision: Open-Ended Versus Closed Questions | |
| |
| |
Organizing Responses to Open-Ended Survey Items: Do You Get Any Satisfaction? | |
| |
| |
Step 1: Asking Respondents' Opinions | |
| |
| |
Step 2: Coding LB/LL Data | |
| |
| |
Example LB/LL: Response Categories | |
| |
| |
Example LB/LL: Participant Responses | |
| |
| |
Step 3: LB/LL Data | |
| |
| |
Example LB/LL: Number of Responses for Each Code | |
| |
| |
Example LB/LL: Participants' Response Pattern | |
| |
| |
Example LB/LL: Summary of Responses | |
| |
| |
Rules for Writing Closed Survey Questions | |
| |
| |
Example: Item-Writing Skills-Length, Clarity, Abbreviations, and Jargon | |
| |
| |
Example: Item-Writing Skills-Concrete Questions | |
| |
| |
Example: Item-Writing Skills-Specificity of Questions | |
| |
| |
Example: Item-Writing Skills-Hidden Biases | |
| |
| |
Example: Item- Writing Skills-Hidden Biases | |
| |
| |
Example: Question-Writing Skills-Very Personal Questions | |
| |
| |
Example: Question-Writing Skills-One Thought per Question | |
| |
| |
Responses for Closed Questions | |
| |
| |
Yes and No | |
| |
| |
Example: Yes-and-No Responses | |
| |
| |
Rating Scales | |
| |
| |
Categorical or Continuous? What about Ordinal? | |
| |
| |
Example: Categorical Rating Scale | |
| |
| |
Example: Ordinal Rating Scale | |
| |
| |
Example: Graphic Rating Scale for Assessing a City Council's Effectiveness | |
| |
| |
Example: Poor Formatting of Graphic Scale | |
| |
| |
Example: Interpreting Graphic Scales | |
| |
| |
Ordinal Scales | |
| |
| |
Example: Ordinal Scale | |
| |
| |
Example: Selecting the Number of Categories | |
| |
| |
Example: Rank Order Scale | |
| |
| |
Example: Comparative Rating Scale | |
| |
| |
Checklist | |
| |
| |
Example: Checklist Responses in which Respondent Must Choose One from a List of Several | |
| |
| |
Example: Checklist Responses that Respondents Answer Yes, No, or Don't Know for Each Item in a List | |
| |
| |
Online Surveys | |
| |
| |
Example: Survey Progress Bar | |
| |
| |
Plain and Simple Survey Questions and Responses | |
| |
| |
Scaling | |
| |
| |
Additive Scales | |
| |
| |
Example: A Survey with an Additive Scale | |
| |
| |
Example: A Survey of Foreign Language Skills | |
| |
| |
Differential Scales | |
| |
| |
Example: Scoring a Differential Scale | |
| |
| |
Summated Scales | |
| |
| |
Example: Creating a Summated Scale for a Self-Esteem Survey | |
| |
| |
Example: Scoring a Summated Scale | |
| |
| |
Summing Up | |
| |
| |
Think About This | |
| |
| |
| |
Getting It Together: Some Practical Concerns | |
| |
| |
Overview | |
| |
| |
Length Counts | |
| |
| |
Example: How a Survey's Circumstances Can Influence Its Length | |
| |
| |
Getting the Survey in Order | |
| |
| |
Example: An Introduction to a Telephone Survey and Its First Question | |
| |
| |
Example: Ordering Survey Questions | |
| |
| |
Example: Ordering Survey Questions | |
| |
| |
Example: Providing Transitions | |
| |
| |
Checklist to Guide Question Order | |
| |
| |
Questionnaire Format: Aesthetics and Other Concerns | |
| |
| |
Response Format | |
| |
| |
Example: Response Formats | |
| |
| |
Branching Questions, or the Infamous "Skip" Pattern | |
| |
| |
Example: Skip Patterns or Branching Questions | |
| |
| |
Administration: Who Gives What to Whom? | |
| |
| |
Self-Administered Questionnaires | |
| |
| |
Checklist for Using Self-Administered Questionnaires | |
| |
| |
Interviews | |
| |
| |
The Survey Is Put on Trial | |
| |
| |
Reliability and Validity: The Quality of Your Survey | |
| |
| |
Is the Survey Reliable? | |
| |
| |
Example: Internal Consistency Counts | |
| |
| |
Example: Internal Consistency Does Not Count | |
| |
| |
Is the Survey Valid? | |
| |
| |
Selecting and Adapting Surveys | |
| |
| |
Example: Excerpt from a Real Online Service Agreement for Free Survey Items (Names Changed) | |
| |
| |
Guidelines for Finding Useable and Useful Surveys in the Research Literature | |
| |
| |
Example: Search for School Dropout Surveys: Sample Findings from the Web of Science | |
| |
| |
Example: Sample Reports of Reliability and Validity in the Research Literature | |
| |
| |
Finding Surveys on the Web | |
| |
| |
Example: Search for Education Surveys: A Sample Page from the National Center for Education Statistics | |
| |
| |
Guidelines for Pilot Testing | |
| |
| |
A Far-Reaching World: Surveys, Language, and Culture | |
| |
| |
Guidelines for Translating Surveys | |
| |
| |
Example: Question about Ethnicity | |
| |
| |
Summing Up | |
| |
| |
Think About This | |
| |
| |
Articles | |
| |
| |
| |
Sampling | |
| |
| |
Overview | |
| |
| |
Sample Size and Response Rate: Who and How Many? | |
| |
| |
Example: Random and Convenience Sampling | |
| |
| |
Random Sampling Methods | |
| |
| |
Example: Simple Random Sampling | |
| |
| |
Example: Not Random Sampling | |
| |
| |
Example: Simple Random Sampling | |
| |
| |
Making the Decision | |
| |
| |
Stratified Random Sampling | |
| |
| |
Example: Stratified Random Sampling | |
| |
| |
Making the Decision | |
| |
| |
Simple Random Cluster Sampling | |
| |
| |
Example: Simple Random Cluster Sampling | |
| |
| |
Making the Decision | |
| |
| |
Systematic Sampling | |
| |
| |
Making the Decision | |
| |
| |
Convenience Samples | |
| |
| |
Making the Decision | |
| |
| |
Other Convenience Sampling Methods | |
| |
| |
Example: Other Convenience Samples | |
| |
| |
Finding the Sample: Who Is In? Who Is Out? | |
| |
| |
How Large Should Your Sample Be? | |
| |
| |
The Standard Error | |
| |
| |
Statistical Methods: Sampling for Two Groups and an Intervention | |
| |
| |
Example: Sample Size Calculations for Sampling Two Groups and an Intervention | |
| |
| |
Subgroups, Measures, Resources, and Schedule | |
| |
| |
Five Questions to Ask When Determining Sample Size | |
| |
| |
Example: Calculating Sample Size in a Survey of Employees in an Experimental and Control Group | |
| |
| |
Example: Power to Detect Differences | |
| |
| |
Example: 80% Power and Effect | |
| |
| |
Example: Sample Size, Effect, and Power | |
| |
| |
Response Rate | |
| |
| |
Tips for Improving Response Rate | |
| |
| |
Weighting | |
| |
| |
Summing Up | |
| |
| |
Think About This | |
| |
| |
Articles | |
| |
| |
| |
Survey Design: Environmental Control | |
| |
| |
Overview | |
| |
| |
Which Designs Are Available? | |
| |
| |
Example: Surveys with Differing Designs | |
| |
| |
Cross-Sectional Survey Designs | |
| |
| |
Example: Cross-Sectional Design | |
| |
| |
Making the Decision | |
| |
| |
Longitudinal Surveys or Cohorts | |
| |
| |
Example: Cohort Design | |
| |
| |
Making the Decision | |
| |
| |
Comparison Group Survey Designs: Quasi- and True Experiments | |
| |
| |
Example: A Quasi-Experimental Design | |
| |
| |
Comparison Group and a Longitudinal Design | |
| |
| |
Example: A True Experimental Comparison Group Design | |
| |
| |
Example: A True Experimental Comparison Group Design and a Longitudinal Design | |
| |
| |
Factorial Designs: Special Cases | |
| |
| |
Other Survey Designs: Normative and Case Control | |
| |
| |
Making the Decision | |
| |
| |
Normative Survey Design | |
| |
| |
Example 1: Normative Design | |
| |
| |
Example 2: Normative Design-Comparison to a Model | |
| |
| |
Making the Decision | |
| |
| |
Case Control Design | |
| |
| |
Making the Decision | |
| |
| |
Example: Case Control Design | |
| |
| |
Survey Design Validity | |
| |
| |
Internal Validity | |
| |
| |
External Validity | |
| |
| |
Surveys, Research Design, and Internal and External Validity | |
| |
| |
Example: Cross-Sectional Survey and Threats to Validity | |
| |
| |
Example: Cohort Design and Threats to Validity | |
| |
| |
Example: Comparison Group Design (True Experiment) and Validity | |
| |
| |
Summing Up | |
| |
| |
Think About This | |
| |
| |
Articles | |
| |
| |
| |
Analyzing and Organizing Data from Surveys | |
| |
| |
Overview | |
| |
| |
What Is Typical Anyway? Some Commonly Used Methods for Analyzing Survey Data | |
| |
| |
Descriptive Statistics | |
| |
| |
Example: Preschool Purposes Questionnaire | |
| |
| |
Example: Frequency of Questionnaire Responses | |
| |
| |
Example: Grouped Ratings of Preschool Purposes by 50 Directors | |
| |
| |
Averages: Means, Medians, and Modes | |
| |
| |
Example: Computing the Median for an Even Number of Scores | |
| |
| |
Example: Computing the Median for an Odd Number of Scores | |
| |
| |
Variation: Range, Variance, and Standard Deviation | |
| |
| |
Correlation and Regression | |
| |
| |
Example: Rank Order Correlation | |
| |
| |
Differences between Groups | |
| |
| |
Statistical Differences | |
| |
| |
Statistical Significance | |
| |
| |
Surveying Differences: Usual Methods | |
| |
| |
Chi-Square | |
| |
| |
Example: Chi-Square | |
| |
| |
The t Test | |
| |
| |
Example: t Test | |
| |
| |
The Mann-Whitney U Test | |
| |
| |
Example: Mann-Whitney U Test | |
| |
| |
ANOVA | |
| |
| |
Risks and Odds | |
| |
| |
Example: Odds Ratio and Relative Risk | |
| |
| |
To Be or Not to Be: Statistician or Qualitative Analyst? | |
| |
| |
Content Analysis, Open-Ended Responses, and Comments | |
| |
| |
Hypothetical Content Analysis: Teasing Boys and Girls | |
| |
| |
Putting the Horse in Front of the Cart: Selecting Analysis Methods | |
| |
| |
Data Management | |
| |
| |
Creating a Code Book or Operational Manual | |
| |
| |
Example: Excerpt from the CARPS, a Survey to Detect Binge Drinking in College Students | |
| |
| |
Example: Excerpt from the Code Book for the CARPS, a Survey to Detect Binge Drinking | |
| |
| |
Establishing Reliable Coding | |
| |
| |
Measuring Agreement: The Kappa | |
| |
| |
Measuring Agreement between Two Coders: The Kappa Statistic | |
| |
| |
Reviewing Surveys for Missing Data | |
| |
| |
Entering the Data | |
| |
| |
Example: Survey Responses from Six People | |
| |
| |
Cleaning the Data | |
| |
| |
Validating Survey Data | |
| |
| |
Summing Up | |
| |
| |
Think About This | |
| |
| |
| |
Presenting the Survey Results | |
| |
| |
Overview | |
| |
| |
Reproducing the Questionnaire | |
| |
| |
Example: Reporting Results with the Questionnaire | |
| |
| |
Example: Online Survey Statistics in Real Time | |
| |
| |
Using Tables | |
| |
| |
Example: Shell Table Describing Children in Two Schools | |
| |
| |
Example: Shell Table for Comparing Children in Two Schools | |
| |
| |
Some Table Preparation Rules | |
| |
| |
Drawing Pie Diagrams | |
| |
| |
Using Bar Graphs | |
| |
| |
Using Line Graphs | |
| |
| |
Example: Downloaded Results of a Customer Satisfaction Survey | |
| |
| |
Example: Transferring the Results of One Question into Bar and Line Graphs | |
| |
| |
Drawing Diagrams or Pictures | |
| |
| |
Example: Words and Diagrams in Survey Reports | |
| |
| |
Writing the Results of a Survey | |
| |
| |
Organizing the Report | |
| |
| |
Example: Structured Abstract of a Survey Report | |
| |
| |
Clear-Writing Tips | |
| |
| |
The Oral Presentation | |
| |
| |
Slide Presentations | |
| |
| |
Oral versus Written Reports: A Difference in Conversation | |
| |
| |
Example: Table Used in a Written and an Oral Report | |
| |
| |
Written Interpretation of the Table | |
| |
| |
Oral Interpretation of the Table | |
| |
| |
Posters | |
| |
| |
Summing Up | |
| |
| |
Think About This | |
| |
| |
Index | |