| |
| |
About the Authors | |
| |
| |
| |
Preparing for an Evaluation | |
| |
| |
| |
Becoming an Accountable Practitioner | |
| |
| |
Evaluation and Accountability | |
| |
| |
The Council on Social Work Education | |
| |
| |
The National Association of Social Workers | |
| |
| |
Quality Improvement in Social Service Programs | |
| |
| |
Case-Level Evaluation | |
| |
| |
Program-Level Evaluation | |
| |
| |
Case-Level and Program-Level Data for Quality Improvement | |
| |
| |
Evaluation and the Profession | |
| |
| |
Contributing to Evidence-Based Practice | |
| |
| |
Collaborating With Program Stakeholders | |
| |
| |
Policymakers | |
| |
| |
General Public | |
| |
| |
Funders | |
| |
| |
Administrators | |
| |
| |
Practitioners | |
| |
| |
Clients | |
| |
| |
A Word About Collaboration Among Stakeholder Groups | |
| |
| |
Integrated Accountability With Service Delivery | |
| |
| |
Client-Centered Practice | |
| |
| |
Evaluation From a Person-in-Environment Perspective | |
| |
| |
Evaluation From a Program-in-Environment Perspective | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Approaches to Accountability | |
| |
| |
The External Project Approach | |
| |
| |
Characteristics of the External Project Approach | |
| |
| |
Externally Driven | |
| |
| |
Resistant Social Workers | |
| |
| |
Intrusiveness | |
| |
| |
Periodic or No Feedback to Social Workers | |
| |
| |
Large Recommended Changes | |
| |
| |
Not Practical in Applied Settings | |
| |
| |
Difficult to Incorporate in Practice Settings | |
| |
| |
The Internal Monitoring Approach | |
| |
| |
Characteristics of the Internal Monitoring Approach | |
| |
| |
Internally Driven | |
| |
| |
Cooperative Social Workers | |
| |
| |
Integrated | |
| |
| |
Ongoing Continuous Feedback | |
| |
| |
Minor Recommended Change | |
| |
| |
Easy to Incorporate in Practice Settings | |
| |
| |
Advantages of the Internal Monitoring Approach | |
| |
| |
Increased Understanding of Programs | |
| |
| |
Relevant Feedback | |
| |
| |
Timely Feedback | |
| |
| |
Self-Protection | |
| |
| |
Practitioner and Client Satisfaction | |
| |
| |
Professionalism | |
| |
| |
Fine-Tuning Programs | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Designing Client-Centered Programs | |
| |
| |
Social Service Agencies | |
| |
| |
Agency Mission Statements | |
| |
| |
Agency Goals | |
| |
| |
Requirements for Goals | |
| |
| |
Agency Objectives | |
| |
| |
Social Service Programs | |
| |
| |
An Agency Versus a Program | |
| |
| |
Program Designs | |
| |
| |
Program Goals | |
| |
| |
Unintended Program Results | |
| |
| |
Program Goals Versus Agency Goals | |
| |
| |
Types of Program Objectives | |
| |
| |
Knowledge-Based Objectives | |
| |
| |
Affect-Based Objectives | |
| |
| |
Behavioral-Based Objectives | |
| |
| |
Qualities of Program Objectives | |
| |
| |
Meaningful | |
| |
| |
Specific | |
| |
| |
Measurable | |
| |
| |
Directional | |
| |
| |
Program Versus Practice Objectives | |
| |
| |
| |
Bob's Self-Sufficiency | |
| |
| |
| |
Jane's Job Dissatisfaction | |
| |
| |
Program Activities | |
| |
| |
Program Logic Models | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Getting Ready for an Evaluation | |
| |
| |
Program Scope and Evaluation | |
| |
| |
Planning With Stakeholders | |
| |
| |
Asking Evaluation Questions | |
| |
| |
Mapping Concepts | |
| |
| |
Reviewing the Literature | |
| |
| |
Developing Schedules | |
| |
| |
Tasks | |
| |
| |
Roles | |
| |
| |
Timelines | |
| |
| |
Producing Documentation | |
| |
| |
Identifying Data Needs | |
| |
| |
Focusing Evaluation Efforts | |
| |
| |
Client Demographics | |
| |
| |
Service Statistics | |
| |
| |
Quality Standards | |
| |
| |
Feedback | |
| |
| |
Client Outcomes | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Doing an Evaluation | |
| |
| |
| |
Doing a Needs Assessment | |
| |
| |
What Are Needs Assessments? | |
| |
| |
Social Problems | |
| |
| |
Social Needs | |
| |
| |
Perceived Needs | |
| |
| |
Normative Needs | |
| |
| |
Relative Needs | |
| |
| |
Expressed Needs | |
| |
| |
Program Solutions | |
| |
| |
Steps in Doing a Needs Assessment | |
| |
| |
| |
Focusing the Problem | |
| |
| |
| |
Developing Needs Assessment Questions | |
| |
| |
| |
Identifying Targets for Intervention (Unit of Analysis) | |
| |
| |
Establishing Target Parameters | |
| |
| |
Sampling (Data Sources) | |
| |
| |
| |
Developing a Data Collection Plan | |
| |
| |
Reviewing Existing Reports | |
| |
| |
Secondary Data Analyses | |
| |
| |
Individual Interviews | |
| |
| |
Group Interviews | |
| |
| |
Telephone and Mail Surveys | |
| |
| |
| |
Analyzing and Displaying Data | |
| |
| |
Collecting Quantitative Data | |
| |
| |
Collecting Qualitative Data | |
| |
| |
| |
Disseminating and Communicating Findings | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Doing a Process Evaluation | |
| |
| |
Purposes of Process Evaluations | |
| |
| |
To Improve a Program's Operations | |
| |
| |
To Generate Knowledge for Our Profession | |
| |
| |
To Estimate Cost Efficiency | |
| |
| |
Steps in a Process Evaluation | |
| |
| |
| |
Deciding What Questions to Ask | |
| |
| |
What Is the Program's Background? | |
| |
| |
What Is the Program's Client Profile? | |
| |
| |
What Is the Program's Staff Profile? | |
| |
| |
What Is the Amount of Service Provided to Clients? | |
| |
| |
What Are the Program's Interventions and Activities? | |
| |
| |
What Administrative Supports Are in Place to Support Client Service Delivery? | |
| |
| |
How Satisfied Are the Program's Stakeholders? | |
| |
| |
How Efficient Is the Program? | |
| |
| |
| |
Developing Data Collection Instruments | |
| |
| |
Ease of Use | |
| |
| |
Appropriateness to the Flow of a Program's Operations | |
| |
| |
Design With User Input | |
| |
| |
| |
Developing a Data Collection Monitoring System | |
| |
| |
Number of Cases to Include (Unit of Analysis) | |
| |
| |
Times to Collect the Data | |
| |
| |
Methods for Collecting the Data | |
| |
| |
| |
Scoring and Analyzing Data | |
| |
| |
| |
Developing a Feedback System | |
| |
| |
| |
Disseminating and Communicating Results | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Doing an Outcome Evaluation | |
| |
| |
Purpose of Outcome Evaluations | |
| |
| |
Uses of Outcome Evaluations | |
| |
| |
Improving Program Services to Clients | |
| |
| |
Influencing Decisions? | |
| |
| |
Generating Knowledge for the Profession | |
| |
| |
Steps in Outcome Evaluations | |
| |
| |
| |
Conceptualizing and Operationalizing Program Objectives | |
| |
| |
| |
Operationalizing Variables and Stating the Outcomes | |
| |
| |
| |
Designing a Monitoring System | |
| |
| |
Deciding the Number of Clients to Include (Unit of Analysis) | |
| |
| |
Deciding When Data Will Be Collected | |
| |
| |
Deciding How Data Will Be Collected | |
| |
| |
| |
Analyzing and Displaying Data | |
| |
| |
| |
Developing a Feedback System | |
| |
| |
| |
Disseminating and Communicating Results | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Doing an Efficiency Evaluation | |
| |
| |
Cost Effectiveness Versus Cost Benefit | |
| |
| |
When to Evaluate for Efficiency | |
| |
| |
Steps in Conducting a Cost-Benefit Evaluation | |
| |
| |
| |
Deciding on an Accounting Perspective | |
| |
| |
The Individual Program's Participants' Perspective | |
| |
| |
The Funding Source's Perspective | |
| |
| |
Applying the Procedure | |
| |
| |
| |
Specifying the Cost-Benefit Model | |
| |
| |
Looking at Costs | |
| |
| |
Looking at Benefits | |
| |
| |
Applying the Procedure | |
| |
| |
| |
Determining Costs | |
| |
| |
Direct Costs | |
| |
| |
Indirect Costs | |
| |
| |
Applying the Procedure | |
| |
| |
| |
Determining Benefits | |
| |
| |
Applying the Procedure | |
| |
| |
| |
Adjusting for Present Value | |
| |
| |
Applying the Procedure | |
| |
| |
| |
Completing the Cost-Benefit Analysis | |
| |
| |
Applying the Procedure | |
| |
| |
Cost-Effectiveness Analyses | |
| |
| |
Applying the Procedure | |
| |
| |
A Few Words About Efficiency-Focused Evaluations | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Gathering Data and Making Decisions | |
| |
| |
| |
Measuring Practice and Program Objectives | |
| |
| |
Why Measurement Is Necessary | |
| |
| |
Objectivity | |
| |
| |
Precision | |
| |
| |
Types of Measuring Instruments | |
| |
| |
Rating Scales | |
| |
| |
Graphic Rating Scales | |
| |
| |
Self-Anchored Rating Scales | |
| |
| |
Summated Scales | |
| |
| |
Goal Attainment Scaling (GAS) | |
| |
| |
Creating Practice Objectives From Program Objectives | |
| |
| |
Measurement by the Numbers | |
| |
| |
Standardized Measuring Instruments | |
| |
| |
Is the Measurement Useful? | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Data Sources, Sampling, and Data Collection Methods | |
| |
| |
Data Sources | |
| |
| |
Sampling | |
| |
| |
Probability Sampling | |
| |
| |
Nonprobability Sampling | |
| |
| |
Data Collection Methods | |
| |
| |
Obtaining Existing Data | |
| |
| |
Documents and Reports | |
| |
| |
Data Sets | |
| |
| |
Obtaining New Data | |
| |
| |
Face-to-Face Individual Interviews | |
| |
| |
Surveys | |
| |
| |
Group Interviews | |
| |
| |
Observation | |
| |
| |
Fitting Data Collection to the Program | |
| |
| |
Ease of Use | |
| |
| |
Appropriateness to the Flow of Program Operations | |
| |
| |
Design With User Input | |
| |
| |
Developing a Data Collection Monitoring System | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Developing a Data Information System | |
| |
| |
Staff Members' Roles in Developing a Data Information System | |
| |
| |
Establishing an Organizational Plan | |
| |
| |
Case-Level Data Collection | |
| |
| |
Program-Level Data Collection | |
| |
| |
Data Collection at Intake | |
| |
| |
Data Collection at Client Contact | |
| |
| |
Data Collection at Termination | |
| |
| |
Data Collection to Obtain Feedback | |
| |
| |
Data Management | |
| |
| |
Manual Data Management | |
| |
| |
Computer-Assisted Data Management | |
| |
| |
Reporting | |
| |
| |
A Look to the Future | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Using Graphics to Report Evaluation Data | |
| |
| |
Characteristics of an Effective Graphic | |
| |
| |
Bar Charts | |
| |
| |
Pie Charts | |
| |
| |
Line Graphs | |
| |
| |
Illustrations | |
| |
| |
Photographs | |
| |
| |
Checklist | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Analyzing Qualitative Data | |
| |
| |
Narrative Data | |
| |
| |
The Analysis Process | |
| |
| |
| |
Get to Know Your Data | |
| |
| |
| |
Focus the Analysis | |
| |
| |
Focus by Question or Topic, Time Period, or Event | |
| |
| |
Focus by Case, Individual, or Group | |
| |
| |
| |
Categorize Information | |
| |
| |
Preset Categories | |
| |
| |
Emergent Categories | |
| |
| |
| |
Identify Patterns and Connections Within and Between Categories | |
| |
| |
Within Category Description | |
| |
| |
Larger Categories | |
| |
| |
Relative Importance | |
| |
| |
Relationships | |
| |
| |
| |
Interpretation-Bringing It All Together | |
| |
| |
"Nuts and Bolts" of Narrative Analysis | |
| |
| |
Data Management Tips | |
| |
| |
Check Your Data | |
| |
| |
Add ID Numbers | |
| |
| |
Prepare Data for Analysis | |
| |
| |
Make Copies | |
| |
| |
Identify the Source of All Data | |
| |
| |
Mark Key Themes | |
| |
| |
Define Categories | |
| |
| |
Cut and Sort | |
| |
| |
Make Connections | |
| |
| |
Enhancing the Process | |
| |
| |
Use Several Sources of Data | |
| |
| |
Track Your Choices | |
| |
| |
Involve Others | |
| |
| |
Pitfalls to Avoid | |
| |
| |
Avoid Generalizing | |
| |
| |
Choose Quotes Carefully | |
| |
| |
Address Limitations and Alternative Explanations | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Making Decisions With Data | |
| |
| |
Objective Data | |
| |
| |
Subjective Data | |
| |
| |
Case-Level Decision Making | |
| |
| |
The Engagement and Problem-Definition Phase | |
| |
| |
The Practice Objective Setting Phase | |
| |
| |
The Intervention Phase | |
| |
| |
Deterioration, or No Change | |
| |
| |
Insufficient, or Slow Change | |
| |
| |
Satisfactory Change | |
| |
| |
The Termination and Follow-Up Phase | |
| |
| |
Program-Level Decision Making | |
| |
| |
Process | |
| |
| |
Outcome | |
| |
| |
Problems and Cases | |
| |
| |
Program | |
| |
| |
Agency | |
| |
| |
Using Outcome Monitoring Data in Program-Level Decision Making | |
| |
| |
Acceptable Results | |
| |
| |
Mixed Results | |
| |
| |
Inadequate Results | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Knowing the Contexts of Evaluations | |
| |
| |
| |
Evaluation Politics, Ethics, and Standards | |
| |
| |
Politics of Evaluation | |
| |
| |
Appropriate and Inappropriate Uses of Evaluation | |
| |
| |
Misuses of Evaluation | |
| |
| |
Justifying Decisions Already Made | |
| |
| |
Inappropriate Use of Public Relations | |
| |
| |
Used for Performance Appraisals | |
| |
| |
Fulfilling Funding Requirements | |
| |
| |
Proper Uses of Evaluation | |
| |
| |
Internal Decision Making | |
| |
| |
External Decision Making | |
| |
| |
Political Influences on the Evaluation Process | |
| |
| |
Manipulating the Evaluation Process | |
| |
| |
Misdirecting the Evaluation Process | |
| |
| |
Program Objectives | |
| |
| |
Sample Selection | |
| |
| |
Data Collection Methods | |
| |
| |
Interpretation of Findings | |
| |
| |
Professional Standards for Evaluation | |
| |
| |
Utility | |
| |
| |
Feasibility | |
| |
| |
Propriety | |
| |
| |
Accuracy | |
| |
| |
Principles of Evaluation Practice | |
| |
| |
| |
Evaluation and Service Delivery Activities Should Be Integrated | |
| |
| |
| |
Involve From the Beginning as Many Stakeholder Groups as Possible | |
| |
| |
| |
Involve All Levels of Staff in the Evaluation Process | |
| |
| |
| |
Make Explicit the Purpose of the Evaluation | |
| |
| |
| |
Provide a Balanced Report and Disseminate Early and Regularly | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Culturally Appropriate Evaluations | |
| |
| |
The Impact of Culture | |
| |
| |
Bridging the Culture Gap | |
| |
| |
Cultural Awareness | |
| |
| |
Intercultural Communication | |
| |
| |
Nonverbal | |
| |
| |
Verbal | |
| |
| |
Cultural Frameworks | |
| |
| |
Orientation to Information | |
| |
| |
Decision Making | |
| |
| |
Individualism | |
| |
| |
Tradition | |
| |
| |
Pace of Life | |
| |
| |
Putting It Together: The Practice of Culturally Competent Evaluation | |
| |
| |
Cultural Awareness | |
| |
| |
Intercultural Communication Skills | |
| |
| |
Developing Specific Knowledge About the Culture | |
| |
| |
Adapting Evaluations | |
| |
| |
Working with Stakeholders | |
| |
| |
Adapting Processes | |
| |
| |
Providing Meaningful Products | |
| |
| |
Summing Up and Looking Ahead | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
| |
Writing Grant Proposals | |
| |
| |
Gathering Background Information | |
| |
| |
Program Concept | |
| |
| |
Your Program | |
| |
| |
Program Expenses | |
| |
| |
Components of a Proposal | |
| |
| |
Executive Summary | |
| |
| |
Statement of Need | |
| |
| |
Program Description | |
| |
| |
Objectives | |
| |
| |
Methods, or Interventions | |
| |
| |
Staffing/Administration | |
| |
| |
Evaluation | |
| |
| |
Sustainability | |
| |
| |
Budget | |
| |
| |
Expense Budget | |
| |
| |
Support and Revenue and Statement | |
| |
| |
Budget Narrative | |
| |
| |
Organizational Information | |
| |
| |
Conclusion | |
| |
| |
Letter Proposal | |
| |
| |
What Happens Next? | |
| |
| |
Summing Up | |
| |
| |
Recap and Online Materials | |
| |
| |
Study Questions | |
| |
| |
References, Further Reading, and Resources | |
| |
| |
Credits | |
| |
| |
Index | |