| |
| |
| |
| Introduction: The World of Educational Assessment | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| The Main Areas in Educational Assessment | |
| |
| |
| Basic Principles | |
| |
| |
| Teacher-made Assessments | |
| |
| |
| Externally Prepared Tests | |
| |
| |
| Special Applications | |
| |
| |
| Guidelines from Professional Groups | |
| |
| |
| Purposes of Assessment | |
| |
| |
| Users of Educational Assessment | |
| |
| |
| Current Trends and Emphases | |
| |
| |
| A Little Bit of History | |
| |
| |
| Standards-based Education | |
| |
| |
| Accountability | |
| |
| |
| Legal Activism | |
| |
| |
| Concern for Fairness | |
| |
| |
| Item Response Theory | |
| |
| |
| Performance Assessment | |
| |
| |
| State, National, and International Testing Programs | |
| |
| |
| Computerization | |
| |
| |
| Keeping Perspective | |
| |
| |
| Some More Terminology | |
| |
| |
| Measurement, Testing, Assessment | |
| |
| |
| Standardized Testing | |
| |
| |
| Distinctions Among Types of Tests | |
| |
| |
| High-stakes Testing | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| |
| Statistics: Just a Little Bit | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| Divisions of Statistics | |
| |
| |
| Using the Computer for Statistics | |
| |
| |
| Variables | |
| |
| |
| Organization of Raw Data | |
| |
| |
| Central Tendency | |
| |
| |
| Variability | |
| |
| |
| Shapes of Distributions | |
| |
| |
| z-scores | |
| |
| |
| The Correlation Coefficient and Scattergram | |
| |
| |
| Cautions in Interpreting the Correlation Coefficient | |
| |
| |
| Correlation Does Not Imply Causation | |
| |
| |
| Most Correlations Detect Only Linear Relationships | |
| |
| |
| The Assumption of Equal Scatter | |
| |
| |
| The Correlation Depends on Variability in the Group | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| |
| Reliability: Consistency of Performance | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| Some Examples | |
| |
| |
| Reliability: The Basic Notion | |
| |
| |
| What Factors Affect Reliability: The Sources of Unreliability | |
| |
| |
| Test Scoring | |
| |
| |
| Test Content | |
| |
| |
| Test Administration | |
| |
| |
| Personal Conditions | |
| |
| |
| A Conceptual Framework | |
| |
| |
| Methods of Determining Reliability | |
| |
| |
| Test-Retest Reliability | |
| |
| |
| Interrater and Intrarater Reliability | |
| |
| |
| Alternate Form Reliability | |
| |
| |
| Internal Consistency Reliability | |
| |
| |
| Three Important Conclusions | |
| |
| |
| The Standard Error of Measurement | |
| |
| |
| Confidence Bands | |
| |
| |
| Appropriate Units for SEM | |
| |
| |
| Some Important Distinctions for the Term "Standard Error" | |
| |
| |
| Reliability in Item Response Theory (IRT) | |
| |
| |
| Reliability for Criterion-Referenced Interpretation | |
| |
| |
| Reliability of What? | |
| |
| |
| Some Concluding Examples | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| |
| Validity: What the Test Measures | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| Introduction | |
| |
| |
| Definition of Validity: Refining Our Thinking | |
| |
| |
| Construct Underrepresentation and Construct Irrelevant Variance | |
| |
| |
| Constructs as Targets | |
| |
| |
| The Basic Issue | |
| |
| |
| Face Validity | |
| |
| |
| Content Validity | |
| |
| |
| Application to Achievement Tests | |
| |
| |
| How to Build Content Validity Into an Assessment | |
| |
| |
| How to Check Content Validity of an Existing Test | |
| |
| |
| Instructional Validity | |
| |
| |
| Content Validity in Other Areas | |
| |
| |
| Problems With Content Validity | |
| |
| |
| Criterion-Related Validity | |
| |
| |
| External, Realistic Criterion | |
| |
| |
| Contrasted Groups | |
| |
| |
| Correlations With Other Tests | |
| |
| |
| Special Considerations for Interpreting Criterion-Related Validity | |
| |
| |
| Construct Validity | |
| |
| |
| Effect of Experimental Variables | |
| |
| |
| Developmental Changes | |
| |
| |
| Factor Analysis | |
| |
| |
| Response Processes | |
| |
| |
| Consequential Validity | |
| |
| |
| Test Fairness, Test Bias | |
| |
| |
| Studying Fairness for Classroom Assessments | |
| |
| |
| Validity vs. Accuracy of Norms | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| |
| Norms and Criteria: Interpreting Student Performance | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| The Basic Issue | |
| |
| |
| Norm-referenced and Criterion-referenced Interpretation | |
| |
| |
| The Raw Score | |
| |
| |
| The Correction for Guessing | |
| |
| |
| The Special Case of Theta ([Theta]) | |
| |
| |
| Types of Norms | |
| |
| |
| Percentile Ranks and Percentiles | |
| |
| |
| Strengths and Weaknesses of Percentile Ranks | |
| |
| |
| Standard Scores | |
| |
| |
| T-scores | |
| |
| |
| SATs and GREs | |
| |
| |
| Deviation IQs | |
| |
| |
| Stanines | |
| |
| |
| Normal Curve Equivalents | |
| |
| |
| Multilevel Standard Scores | |
| |
| |
| Strengths and Weaknesses of Standard Scores | |
| |
| |
| Developmental Norms | |
| |
| |
| Grade Equivalents | |
| |
| |
| Mental Age | |
| |
| |
| Other Developmental Norms | |
| |
| |
| Strengths and Weaknesses of Developmental Norms | |
| |
| |
| Examples of Norm Tables | |
| |
| |
| Narrative Reports and Norms | |
| |
| |
| Norm Groups | |
| |
| |
| National Norms | |
| |
| |
| International Norms | |
| |
| |
| Convenience Norm Groups | |
| |
| |
| User Norms | |
| |
| |
| Subgroup Norms | |
| |
| |
| Local Norms | |
| |
| |
| Institutional Norms and School Norms | |
| |
| |
| The Standardization Group: Determining Its Usefulness | |
| |
| |
| Criterion-referenced Interpretation | |
| |
| |
| Setting Performance Standards | |
| |
| |
| Special Scores | |
| |
| |
| Sample Reports | |
| |
| |
| Looking at Dora's Reports | |
| |
| |
| Self-Referencing on Repeated Measures | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| |
| Planning for Assessment | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| Introduction | |
| |
| |
| General Goals vs. Specific Objectives | |
| |
| |
| Taxonomies for Educational Objectives | |
| |
| |
| Bloom's Taxonomy | |
| |
| |
| The Affective Domain | |
| |
| |
| The Psychomotor Domain | |
| |
| |
| Alternative Taxonomies for the Cognitive Domain | |
| |
| |
| A Conclusion and a Caution | |
| |
| |
| Sources of Educational Objectives | |
| |
| |
| Professional Organizations | |
| |
| |
| States | |
| |
| |
| NAEP and TIMSS | |
| |
| |
| Textbooks | |
| |
| |
| School Districts | |
| |
| |
| Teacher-prepared Objectives | |
| |
| |
| Guidelines for Preparing Your Own Objectives | |
| |
| |
| The Overall Assessment Plan | |
| |
| |
| Planning a Specific Assessment | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| |
| Selected-Response Items: Multiple Choice, True-False, and Matching | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| Types of Test Items: Categories and Terms | |
| |
| |
| Comparison of Selected-Response and Constructed-Response Items | |
| |
| |
| Multiple-choice Items | |
| |
| |
| Suggestions for Writing MC Items | |
| |
| |
| A Brief Summary | |
| |
| |
| True-False Items | |
| |
| |
| Other Binary Choice Formats | |
| |
| |
| Matching Items | |
| |
| |
| Allowing Student Comments | |
| |
| |
| Developing Item-Writing Skill | |
| |
| |
| Lucky Guessing | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| |
| Constructed-Response Items: Essays, Performances, and Portfolios | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| General Nature of Constructed-Response Items | |
| |
| |
| Two Key Challenges for Constructed-Response Items | |
| |
| |
| Completion Items | |
| |
| |
| Essays | |
| |
| |
| Performance Tasks | |
| |
| |
| Methods of Scoring and Scoring Rubrics | |
| |
| |
| Automated Scoring | |
| |
| |
| Special Issues for Performance Assessments | |
| |
| |
| Differing Applicability | |
| |
| |
| Collaborative Work on Performance Tasks | |
| |
| |
| Essays and Performance Tasks Completed Outside the Classroom | |
| |
| |
| Brief CRs for Formative Evaluation | |
| |
| |
| Suggestions for Preparing Constructed-Response Items | |
| |
| |
| Suggestions for Completing the Scoring | |
| |
| |
| Portfolio Assessment | |
| |
| |
| Special Characteristics of Portfolios | |
| |
| |
| Some Cautions | |
| |
| |
| Principles for Evaluating Portfolios | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| |
| Interests, Creativity, and Nontest Indicators | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| Assessing Interests and Attitudes | |
| |
| |
| The Likert Method | |
| |
| |
| The Problem of Faking | |
| |
| |
| Free-Response Measures of Interests and Attitudes | |
| |
| |
| Two Practical Matters | |
| |
| |
| Existing Measures of Interests and Attitudes | |
| |
| |
| Assessing Creativity | |
| |
| |
| Convergent and Divergent Thinking | |
| |
| |
| Prompts for Creative Thinking | |
| |
| |
| Scoring Creative Responses | |
| |
| |
| Standardized Tests for Creative Thinking | |
| |
| |
| Nontest Indicators | |
| |
| |
| Follow-up Activities: Jobs and College | |
| |
| |
| Indexes From Routine Records | |
| |
| |
| Satisfaction Reports | |
| |
| |
| Self-Reports of Achievement | |
| |
| |
| Unobtrusive Measures | |
| |
| |
| Nontest Indicators for Classroom Use | |
| |
| |
| Outcomes, Not Inputs | |
| |
| |
| Concluding Notes on Nontest Indicators | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| |
| Administering and Analyzing Tests | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| Administering Tests | |
| |
| |
| Not Fair! | |
| |
| |
| Preparing Test Materials | |
| |
| |
| Remote Preparation of Students | |
| |
| |
| Immediate Preparation of Students | |
| |
| |
| Managing the Testing Environment | |
| |
| |
| Preparation for Scoring | |
| |
| |
| Returning the Tests | |
| |
| |
| Wrapping Up | |
| |
| |
| Item Analysis | |
| |
| |
| Two Key Item Statistics: p and D | |
| |
| |
| How Does Item Analysis Help? | |
| |
| |
| Item Analysis in Large-Scale Testing Programs | |
| |
| |
| Item Analysis for Interest and Attitude Measures | |
| |
| |
| Item Analysis for Essays and Performance Items | |
| |
| |
| Item Analysis Computer Programs | |
| |
| |
| Qualitative, Informal Item Analysis | |
| |
| |
| Cheating: Prevalence, Practices, and Prevention | |
| |
| |
| Prevalence | |
| |
| |
| Practices | |
| |
| |
| Prevention | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| |
| Standardized Tests I: Achievement | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| Standardized Achievement Tests: What Are They? | |
| |
| |
| A Broad Classification of Standardized Achievement Tests | |
| |
| |
| Some Common Characteristics | |
| |
| |
| Contrasts with Teacher-Made Tests | |
| |
| |
| Achievement Batteries | |
| |
| |
| An Example: Stanford Achievement Test | |
| |
| |
| A Class List Report for Ms. Prescott's Class | |
| |
| |
| Typical Uses and Special Features | |
| |
| |
| Single-area Achievement Tests | |
| |
| |
| Examples | |
| |
| |
| Looking at Jenny's Report | |
| |
| |
| Diagnostic Tests | |
| |
| |
| Typical Uses and Special Features | |
| |
| |
| Licensing and Certification Tests | |
| |
| |
| Examples | |
| |
| |
| The Praxis Exams | |
| |
| |
| Typical Uses and Special Features | |
| |
| |
| State Assessment Programs | |
| |
| |
| General Characteristics | |
| |
| |
| Sources of Information about State Assessment Programs | |
| |
| |
| A National and an International Testing Program | |
| |
| |
| A National Testing Program: NAEP | |
| |
| |
| An International Testing Program: TIMSS | |
| |
| |
| Special Features | |
| |
| |
| Individually Administered Achievement Tests | |
| |
| |
| Examples | |
| |
| |
| Typical Uses and Special Features | |
| |
| |
| Administering Standardized Tests | |
| |
| |
| Before Testing | |
| |
| |
| During the Test | |
| |
| |
| After the Test | |
| |
| |
| Clearly Unethical Practices | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| |
| Standardized Tests II: Ability, Interests, and Personality | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| Mental Ability Tests | |
| |
| |
| A Profusion of Names | |
| |
| |
| Theories About Mental Ability | |
| |
| |
| Individually Administered Mental Ability Tests | |
| |
| |
| General Features of Individually Administered Mental Ability Tests | |
| |
| |
| Examples of Individually Administered Mental Ability Tests | |
| |
| |
| Group-Administered Mental Ability Tests | |
| |
| |
| General Features of Group-Administered Mental Ability Tests | |
| |
| |
| Examples of Group-Administered Mental Ability Tests | |
| |
| |
| Intelligence Tests and the Definition of Mental Retardation | |
| |
| |
| Intelligence Tests and the Definition of Learning Disabilities | |
| |
| |
| Personality and Behavior Rating Scales | |
| |
| |
| Self-Report Inventories | |
| |
| |
| Projective Techniques | |
| |
| |
| Behavior Rating Scales | |
| |
| |
| Career Interest Inventories | |
| |
| |
| General Characteristics | |
| |
| |
| Typical Uses and Users | |
| |
| |
| Widely Used Career Interest Inventories | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| |
| Grading and Reporting | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| Definition of Grading | |
| |
| |
| Purposes of Grades | |
| |
| |
| Feedback | |
| |
| |
| Administrative Purposes | |
| |
| |
| Motivation | |
| |
| |
| Rationales for Assigning Grades | |
| |
| |
| Pros and Cons of the Various Rationales | |
| |
| |
| Research Results and Conclusions | |
| |
| |
| Coding Systems: The Actual Grades | |
| |
| |
| Letter Grades | |
| |
| |
| Percentage Grades | |
| |
| |
| Verbal Descriptors | |
| |
| |
| Checklists | |
| |
| |
| Narrative Reports | |
| |
| |
| Combining Information to Get a Final Grade | |
| |
| |
| Record Keeping | |
| |
| |
| Applying Weights to the Elements | |
| |
| |
| Converting All Scores to a Common Scale | |
| |
| |
| Converting the Composite to a Grade | |
| |
| |
| Three Special Problems | |
| |
| |
| The Effect of Variability on Weights | |
| |
| |
| The Effect of Zero Scores | |
| |
| |
| The Effect of Regression to the Mean | |
| |
| |
| Grades for Nonacademic Areas | |
| |
| |
| Report Cards | |
| |
| |
| Other Procedures for Reporting to Parents | |
| |
| |
| Parent-Teacher Conference | |
| |
| |
| Reporting Standardized Test Results to Parents | |
| |
| |
| Other Communications | |
| |
| |
| Legal Considerations | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| |
| Educational Assessment and the Law | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| Definition of Laws | |
| |
| |
| Basic Terminology | |
| |
| |
| Why Is It So Confusing? | |
| |
| |
| Laws Related to Assessment | |
| |
| |
| The 14th Amendment | |
| |
| |
| The Civil Rights Acts of 1964 and 1991 | |
| |
| |
| Elementary and Secondary Education Act of 1965 | |
| |
| |
| Rehabilitation Act of 1973, and the Americans With Disabilities Act of 1990 | |
| |
| |
| Individuals With Disabilities Education Act: IDEA 2004 and Its Predecessors | |
| |
| |
| No Child Left Behind Act | |
| |
| |
| Family Educational Rights and Privacy Act | |
| |
| |
| Gifted and Talented Students | |
| |
| |
| Illustrative Court Cases | |
| |
| |
| Debra P. v. Turlington and GI Forum v. TEA | |
| |
| |
| Larry P. v. Riles, PASE v. Hannon, and Crawford v. Honig | |
| |
| |
| A Concluding Reminder on Grading | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| |
| Evaluating Teaching: Applying Assessment to Yourself | |
| |
| |
| Objectives | |
| |
| |
| What This Chapter Is About | |
| |
| |
| Why This Topic Is Important | |
| |
| |
| Our Focus of Attention | |
| |
| |
| Let's Remember Basic Principles | |
| |
| |
| Why Is It So Difficult? | |
| |
| |
| Administrator Ratings | |
| |
| |
| Mentor/Colleague Ratings | |
| |
| |
| Videotaping and Audio Recording | |
| |
| |
| Protfolios | |
| |
| |
| Student Test Scores | |
| |
| |
| Student Ratings of Teachers | |
| |
| |
| Parent Ratings | |
| |
| |
| Student Self-Reports of Achievement | |
| |
| |
| Unobtrusive Measures | |
| |
| |
| Teacher Competency Tests and Additional Certifications | |
| |
| |
| Don't Forget the "Other Stuff" | |
| |
| |
| The Importance of Content | |
| |
| |
| From Information to Reflection to Action | |
| |
| |
| Practical Advice | |
| |
| |
| Summary | |
| |
| |
| Key Terms | |
| |
| |
| Exercises | |
| |
| |
| Appendixes | |
| |
| |
| |
| Sources of Information About Tests | |
| |
| |
| |
| Major Test Publishers | |
| |
| |
| |
| Useful Formulas | |
| |
| |
| |
| Sample IEP | |
| |
| |
| |
| State Web Sites | |
| |
| |
| Glossary | |
| |
| |
| References | |
| |
| |
| Name Index | |
| |
| |
| Subject Index | |