| |
| |
| Preface | |
| |
| |
| Acknowledgments | |
| |
| |
| About the Authors | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Why This Book? | |
| |
| |
| |
| What Is Usability Testing? | |
| |
| |
| |
| The Importance of Moderating Skills | |
| |
| |
| |
| The Golden Rules of Moderating | |
| |
| |
| |
| Cultural Points of View | |
| |
| |
| |
| About the Sidebars in This Book | |
| |
| |
| |
| About the Videos that Accompany This Book | |
| |
| |
| |
| About the Companion Web Site | |
| |
| |
| |
| Getting started as a test moderator | |
| |
| |
| |
| What Makes a Great Moderator? | |
| |
| |
| |
| Understanding usability testing | |
| |
| |
| |
| The basics of interacting | |
| |
| |
| |
| The ability to establish and maintain a rapport | |
| |
| |
| |
| Lots of practice | |
| |
| |
| |
| Roles of a Moderator | |
| |
| |
| |
| The Gracious Host | |
| |
| |
| |
| The Leader | |
| |
| |
| |
| The Neutral Observer | |
| |
| |
| |
| Other possible roles | |
| |
| |
| |
| When roles change | |
| |
| |
| |
| When roles conflict | |
| |
| |
| |
| Testing Locations | |
| |
| |
| |
| Test Preparation | |
| |
| |
| |
| Planning ahead | |
| |
| |
| |
| Planning for many tasks | |
| |
| |
| |
| Understanding the domain and product | |
| |
| |
| |
| Jump-Starting Your Moderating Skills | |
| |
| |
| |
| Six things you can do first | |
| |
| |
| |
| The big challenges | |
| |
| |
| |
| Golden rules 1 through 5 | |
| |
| |
| |
| Rule 1: Decide How to Interact Based on the Purpose of the Test | |
| |
| |
| |
| The type of test | |
| |
| |
| |
| The product's stage of development | |
| |
| |
| |
| Relationship with developers | |
| |
| |
| |
| Rule 2: Protect Participants' Rights | |
| |
| |
| |
| Compensation | |
| |
| |
| |
| Informed consent | |
| |
| |
| |
| Confidentiality | |
| |
| |
| |
| Balance of purpose and risk | |
| |
| |
| |
| Priorities | |
| |
| |
| |
| Rule 3: Remember Your Responsibility to Future Users | |
| |
| |
| |
| Letting participants struggle | |
| |
| |
| |
| Conflicting responsibilities | |
| |
| |
| |
| Rule 4: Respect the Participants as Experts, but Remain in Charge | |
| |
| |
| |
| Controlling logistics and pacing | |
| |
| |
| |
| Stopping unacceptable behavior | |
| |
| |
| |
| Dealing with uncertainty | |
| |
| |
| |
| Projecting authority | |
| |
| |
| |
| Dispelling negativity | |
| |
| |
| |
| Managing visitors | |
| |
| |
| |
| Rule 5: Be Professional, Which Includes Being Genuine | |
| |
| |
| |
| Dos for making and maintaining a connection | |
| |
| |
| |
| Don'ts for making and maintaining a connection | |
| |
| |
| |
| When you're not in the room | |
| |
| |
| |
| Golden rules 6 through 10 | |
| |
| |
| |
| Rule 6: Let the Participants Speak! | |
| |
| |
| |
| Speakership | |
| |
| |
| |
| Appropriate interruptions | |
| |
| |
| |
| Judicious speaking | |
| |
| |
| |
| Silent communication | |
| |
| |
| |
| Rule 7: Remember That Your Intuition Can Hurt and Help You | |
| |
| |
| |
| Your intuition can hurt you | |
| |
| |
| |
| Your intuition can help you | |
| |
| |
| |
| Rule 8: Be Unbiased | |
| |
| |
| |
| Use an unbiased test script | |
| |
| |
| |
| Use unbiased questions | |
| |
| |
| |
| Keep answers unbiased | |
| |
| |
| |
| Watch nonverbal cues | |
| |
| |
| |
| Rule 9: Don't Give Away Information Inadvertently | |
| |
| |
| |
| Giving an assist | |
| |
| |
| |
| Explaining the designer's intent or being defensive | |
| |
| |
| |
| Recording all suggestions | |
| |
| |
| |
| Rule 10: Watch Yourself to Keep Sharp | |
| |
| |
| |
| Initial contacts | |
| |
| |
| |
| Recruiting | |
| |
| |
| |
| Contacting participants | |
| |
| |
| |
| Advertising | |
| |
| |
| |
| Explaining the test | |
| |
| |
| |
| Screening candidates | |
| |
| |
| |
| Confirming appointments | |
| |
| |
| |
| When Participants Arrive | |
| |
| |
| |
| Greeting participants | |
| |
| |
| |
| Creating comfort | |
| |
| |
| |
| Obtaining informed consent | |
| |
| |
| |
| The Pretest Briefing | |
| |
| |
| |
| Preparing yourself | |
| |
| |
| |
| Preparing participants | |
| |
| |
| |
| Using a script or checklist | |
| |
| |
| |
| Practicing the think-aloud technique | |
| |
| |
| |
| Confirming that participants are ready | |
| |
| |
| |
| Transitioning to the Tasks | |
| |
| |
| |
| Starting the tasks | |
| |
| |
| |
| conducting a pretest interview | |
| |
| |
| |
| Interacting during the session | |
| |
| |
| |
| Interacting for a Reason | |
| |
| |
| |
| Keeping Them Talking | |
| |
| |
| |
| Prompting as a reminder | |
| |
| |
| |
| Prompting the silent ones | |
| |
| |
| |
| When and How to Probe | |
| |
| |
| |
| Probing questions | |
| |
| |
| |
| Planned versus spontaneous probes | |
| |
| |
| |
| Probes to avoid | |
| |
| |
| |
| Common probes | |
| |
| |
| |
| Providing Encouragement | |
| |
| |
| |
| Encouraging statements to avoid | |
| |
| |
| |
| Common encouraging statements | |
| |
| |
| |
| Dealing with Failure | |
| |
| |
| |
| Participants' self-blame | |
| |
| |
| |
| The moderator's distress | |
| |
| |
| |
| The participant's distress | |
| |
| |
| |
| Responses to participants' distress | |
| |
| |
| |
| Stopping a test | |
| |
| |
| |
| What you shouldn't do when a participant fails | |
| |
| |
| |
| Providing Assistance | |
| |
| |
| |
| An assist | |
| |
| |
| |
| Giving assistance | |
| |
| |
| |
| Interventions versus assists | |
| |
| |
| |
| Not giving assistance | |
| |
| |
| |
| Levels of assistance | |
| |
| |
| |
| Completing a task for a participant | |
| |
| |
| |
| Measuring assists | |
| |
| |
| |
| Interacting during post-test activities | |
| |
| |
| |
| Maintaining Your Roles | |
| |
| |
| |
| Determining the Order of Activities | |
| |
| |
| |
| Clarifying Things That Occurred during the Test | |
| |
| |
| |
| Administering Ratings and Questionnaires | |
| |
| |
| |
| Questionnaires | |
| |
| |
| |
| Reasons behind the ratings | |
| |
| |
| |
| Accuracy of ratings | |
| |
| |
| |
| Asking Open-Ended Questions | |
| |
| |
| |
| Allowing Others to Interact with Participants | |
| |
| |
| |
| Final Activities | |
| |
| |
| |
| Providing incentives | |
| |
| |
| |
| Ending the session | |
| |
| |
| |
| Interacting in a remote test session | |
| |
| |
| |
| What Is Remote Testing? | |
| |
| |
| |
| Synchronous and asynchronous testing | |
| |
| |
| |
| The technology | |
| |
| |
| |
| Advantages and disadvantages | |
| |
| |
| |
| Preparing for the Session | |
| |
| |
| |
| Recruiting | |
| |
| |
| |
| Obtaining informed consent and confidentiality | |
| |
| |
| |
| Determining the equipment candidates have | |
| |
| |
| |
| Establishing willingness and downloading software | |
| |
| |
| |
| Interacting during the Session | |
| |
| |
| |
| Establishing what participants see on their screen | |
| |
| |
| |
| Providing instructions on thinking aloud | |
| |
| |
| |
| Making the task scenarios available to participants | |
| |
| |
| |
| Avoiding dependencies between tasks | |
| |
| |
| |
| Managing visitors during the session | |
| |
| |
| |
| Dealing with distractions at the participants' end | |
| |
| |
| |
| Making a connection despite the physical distance | |
| |
| |
| |
| Moderator-participant arrangements | |
| |
| |
| |
| A Bit of History | |
| |
| |
| |
| Physical Arrangement | |
| |
| |
| |
| The moderator in the test room | |
| |
| |
| |
| The moderator not in the test room | |
| |
| |
| |
| Beliefs about Arrangements | |
| |
| |
| |
| Physical separation | |
| |
| |
| |
| Physical proximity | |
| |
| |
| |
| Choice of Arrangement | |
| |
| |
| |
| Being physically close to participants | |
| |
| |
| |
| Being physically separated from participants | |
| |
| |
| |
| Considerations for the Practitioner | |
| |
| |
| |
| Interacting with diverse populations | |
| |
| |
| |
| General Guidelines | |
| |
| |
| |
| People with Physical Disabilities | |
| |
| |
| |
| Interacting with the physically disabled | |
| |
| |
| |
| Interacting with blind participants | |
| |
| |
| |
| Interacting with deaf and hard-of-hearing participants | |
| |
| |
| |
| The Elderly | |
| |
| |
| |
| Recruiting elders | |
| |
| |
| |
| Interacting with elders | |
| |
| |
| |
| People Who Have Low Literacy Skills | |
| |
| |
| |
| Functional illiteracy | |
| |
| |
| |
| Interacting with functionally illiterate participants | |
| |
| |
| |
| Testing with low-literacy participants | |
| |
| |
| |
| Children and Teens | |
| |
| |
| |
| Grouping children by age | |
| |
| |
| |
| Recruiting children and teens | |
| |
| |
| |
| Interacting during the session | |
| |
| |
| |
| People From Other Cultures | |
| |
| |
| |
| Interacting with participants from the same and other cultures | |
| |
| |
| |
| Adapting your techniques | |
| |
| |
| |
| Integrating the videos | |
| |
| |
| |
| About the Videos | |
| |
| |
| |
| The test session videos | |
| |
| |
| |
| Use of the videos | |
| |
| |
| |
| Content of the Videos | |
| |
| |
| |
| Video 1: Pretest briefing with a checklist | |
| |
| |
| |
| Video 2: Pretest briefing following a script | |
| |
| |
| |
| Video 3: Interacting during the session, example 1 | |
| |
| |
| |
| Video 4: Interacting during the session, example 2 | |
| |
| |
| |
| Video 5: Interacting in a remote testing situation | |
| |
| |
| |
| Video 6: Post-test interview | |
| |
| |
| |
| The Future of Usability Testing | |
| |
| |
| |
| Current trends | |
| |
| |
| |
| What's next? | |
| |
| |
| References | |
| |
| |
| Index | |