| |
| |
| Preface to the Second Edition | |
| |
| |
| Acknowledgments | |
| |
| |
| Biographies | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| What Is User Experience | |
| |
| |
| |
| What Are User Experience Metrics? | |
| |
| |
| |
| The Value of UX Metrics | |
| |
| |
| |
| Metrics for Everyone | |
| |
| |
| |
| New Technologies in UX Metrics | |
| |
| |
| |
| Ten Myths about UX Metrics | |
| |
| |
| |
| Metrics Take Too Much Time to Collect | |
| |
| |
| |
| UX Metrics Cost Too Much Money | |
| |
| |
| |
| UX Metrics Are Not Useful When Focusing on Small Improvements | |
| |
| |
| |
| UX Metrics Don't Help Us Understand Causes | |
| |
| |
| |
| UX Metrics Are Too Noisy | |
| |
| |
| |
| You Can Just Trust Your Gut | |
| |
| |
| |
| Metrics Don't Apply to New Products | |
| |
| |
| |
| No Metrics Exist for the Type of Issues We Are Dealing with | |
| |
| |
| |
| Metrics Are not Understood or Appreciated by Management | |
| |
| |
| |
| It's Difficult to Collect Reliable Data with a Small Sample Size | |
| |
| |
| |
| Background | |
| |
| |
| |
| Independent and Dependent Variables | |
| |
| |
| |
| Types of Data | |
| |
| |
| |
| Nominal Data | |
| |
| |
| |
| Ordinal Data | |
| |
| |
| |
| Interval Data | |
| |
| |
| |
| Ratio Data | |
| |
| |
| |
| Descriptive Statistics | |
| |
| |
| |
| Measures of Central Tendency | |
| |
| |
| |
| Measures of Variability | |
| |
| |
| |
| Confidence Intervals | |
| |
| |
| |
| Displaying Confidence Intervals as Error Bars | |
| |
| |
| |
| Comparing Means | |
| |
| |
| |
| Independent Samples | |
| |
| |
| |
| Paired Samples | |
| |
| |
| |
| Comparing More Than Two Samples | |
| |
| |
| |
| Relationships Between Variables | |
| |
| |
| |
| Correlations | |
| |
| |
| |
| Nonparametric Tests | |
| |
| |
| |
| The x<sup>2</sup> Test | |
| |
| |
| |
| Presenting your Data Graphically | |
| |
| |
| |
| Column or Bar Graphs | |
| |
| |
| |
| Line Graphs | |
| |
| |
| |
| Scatterplots | |
| |
| |
| |
| Pie or Donut Charts | |
| |
| |
| |
| Stacked Bar or Column Graphs | |
| |
| |
| |
| Summary | |
| |
| |
| |
| Planning | |
| |
| |
| |
| Study Goals | |
| |
| |
| |
| Formative Usability | |
| |
| |
| |
| Summative Usability | |
| |
| |
| |
| User Goals | |
| |
| |
| |
| Performance | |
| |
| |
| |
| Satisfaction | |
| |
| |
| |
| Choosing the Right Metrics: Ten Types of Usability Studies | |
| |
| |
| |
| Completing a Transaction | |
| |
| |
| |
| Comparing Products | |
| |
| |
| |
| Evaluating Frequent Use of the Same Product | |
| |
| |
| |
| Evaluating Navigation and/or Information Architecture | |
| |
| |
| |
| Increasing Awareness | |
| |
| |
| |
| Problem Discovery | |
| |
| |
| |
| Maximizing Usability for a Critical Product | |
| |
| |
| |
| Creating an Overall Positive User Experience | |
| |
| |
| |
| Evaluating the Impact of Subtle Changes | |
| |
| |
| |
| Comparing Alternative Designs | |
| |
| |
| |
| Evaluation Methods | |
| |
| |
| |
| Traditional (Moderated) Usability Tests | |
| |
| |
| |
| Online (Unmoderated) Usability Tests | |
| |
| |
| |
| Online Surveys | |
| |
| |
| |
| Other Study Details | |
| |
| |
| |
| Budgets and Timelines | |
| |
| |
| |
| Participants | |
| |
| |
| |
| Data Collection | |
| |
| |
| |
| Data Cleanup | |
| |
| |
| |
| Summary | |
| |
| |
| |
| Performance Metrics | |
| |
| |
| |
| Task Success | |
| |
| |
| |
| Binary Success | |
| |
| |
| |
| Levels of Success | |
| |
| |
| |
| Issues in Measuring Success | |
| |
| |
| |
| Time on Task | |
| |
| |
| |
| Importance of Measuring Time on Task | |
| |
| |
| |
| How to Collect and Measure Time on Task | |
| |
| |
| |
| Analyzing and Presenting Time-on-Task Data | |
| |
| |
| |
| Issues to Consider When Using Time Data | |
| |
| |
| |
| Errors | |
| |
| |
| |
| When to Measure Errors | |
| |
| |
| |
| What Constitutes an Error? | |
| |
| |
| |
| Collecting and Measuring Errors | |
| |
| |
| |
| Analyzing and Presenting Errors | |
| |
| |
| |
| Issues to Consider When Using Error Metrics | |
| |
| |
| |
| Efficiency | |
| |
| |
| |
| Collecting and Measuring Efficiency | |
| |
| |
| |
| Analyzing and Presenting Efficiency Data | |
| |
| |
| |
| Efficiency as a Combination of Task Success and Time | |
| |
| |
| |
| Learnability | |
| |
| |
| |
| Collecting and Measuring Learnability Data | |
| |
| |
| |
| Analyzing and Presenting Learnability Data | |
| |
| |
| |
| Issues to Consider When Measuring Learnability | |
| |
| |
| |
| Summary | |
| |
| |
| |
| Issue-Based Metrics | |
| |
| |
| |
| What Is a Usability Issue? | |
| |
| |
| |
| Real Issues versus False Issues | |
| |
| |
| |
| How to Identify an Issue | |
| |
| |
| |
| In-Person Studies | |
| |
| |
| |
| Automated Studies | |
| |
| |
| |
| Severity Ratings | |
| |
| |
| |
| Severity Ratings Based on the User Experience | |
| |
| |
| |
| Severity Ratings Based on a Combination of Factors | |
| |
| |
| |
| Using a Severity Rating System | |
| |
| |
| |
| Some Caveats about Rating Systems | |
| |
| |
| |
| Analyzing and Reporting Metrics for Usability Issues | |
| |
| |
| |
| Frequency of Unique Issues | |
| |
| |
| |
| Frequency of Issues Per Participant | |
| |
| |
| |
| Frequency of Participants | |
| |
| |
| |
| Issues by Category | |
| |
| |
| |
| Issues by Task | |
| |
| |
| |
| Consistency in Identifying Usability Issues | |
| |
| |
| |
| Bias in Identifying Usability Issues | |
| |
| |
| |
| Number of Participants | |
| |
| |
| |
| Five Participants Is Enough | |
| |
| |
| |
| Five Participants Is Not Enough | |
| |
| |
| |
| Our Recommendation | |
| |
| |
| |
| Summary | |
| |
| |
| |
| Self-Reported Metrics | |
| |
| |
| |
| Importance of Self-Reported Data | |
| |
| |
| |
| Rating Scales | |
| |
| |
| |
| Likert Scales | |
| |
| |
| |
| Semantic Differential Scales | |
| |
| |
| |
| When to Collect Self-Reported Data | |
| |
| |
| |
| How to Collect Ratings | |
| |
| |
| |
| Biases in Collecting Self-Reported Data | |
| |
| |
| |
| General Guidelines for Rating Scales | |
| |
| |
| |
| Analyzing Rating-Scale Data | |
| |
| |
| |
| Post-Task Ratings | |
| |
| |
| |
| Ease of Use | |
| |
| |
| |
| After-Scenario Questionnaire (ASQ) | |
| |
| |
| |
| Expectation Measure | |
| |
| |
| |
| A Comparison of Post-task Self-Reported Metrics | |
| |
| |
| |
| Postsession Ratings | |
| |
| |
| |
| Aggregating Individual Task Ratings | |
| |
| |
| |
| System Usability Scale | |
| |
| |
| |
| Computer System Usability Questionnaire | |
| |
| |
| |
| Questionnaire for User Interface Satisfaction | |
| |
| |
| |
| Usefulness, Satisfaction, and Ease-of-Use Questionnaire | |
| |
| |
| |
| Product Reaction Cards | |
| |
| |
| |
| A Comparison of Postsession Self-Reported Metrics | |
| |
| |
| |
| Net Promoter Score | |
| |
| |
| |
| Using SUS to Compare Designs | |
| |
| |
| |
| Online Services | |
| |
| |
| |
| Website Analysis and Measurement Inventory | |
| |
| |
| |
| American Customer Satisfaction Index | |
| |
| |
| |
| OpinionLab | |
| |
| |
| |
| Issues with Live-Site Surveys | |
| |
| |
| |
| Other Types of Self-Reported Metrics | |
| |
| |
| |
| Assessing Specific Attributes | |
| |
| |
| |
| Assessing Specific Elements | |
| |
| |
| |
| Open-Ended Questions | |
| |
| |
| |
| Awareness and Comprehension | |
| |
| |
| |
| Awareness and Usefulness Gaps | |
| |
| |
| |
| Summary | |
| |
| |
| |
| Behavioral and Physiological Metrics | |
| |
| |
| |
| Observing and Coding Unprompted Verbal Expressions | |
| |
| |
| |
| Eye Tracking | |
| |
| |
| |
| How Eye Tracking Works | |
| |
| |
| |
| Visualizing Eye-Tracking Data | |
| |
| |
| |
| Areas of Interest | |
| |
| |
| |
| Common Eye-Tracking Metrics | |
| |
| |
| |
| Eye-Tracking Analysis Tips | |
| |
| |
| |
| Pupillary Response | |
| |
| |
| |
| Measuring Emotion | |
| |
| |
| |
| Affectiva and the Q-Sensor | |
| |
| |
| |
| Blue Bubble Lab and Emovision | |
| |
| |
| |
| Seren and Emotiv | |
| |
| |
| |
| Stress and Other Physiological Measures | |
| |
| |
| |
| Heart Rate Variance | |
| |
| |
| |
| Heart Rate Variance and Skin Conductance Research | |
| |
| |
| |
| Other Measures | |
| |
| |
| |
| Summary | |
| |
| |
| |
| Combined and Comparative Metrics | |
| |
| |
| |
| Single Usability Scores | |
| |
| |
| |
| Combining Metrics Based on Target Goals | |
| |
| |
| |
| Combining Metrics Based on Percentages | |
| |
| |
| |
| Combining Metrics Based on Z Scores | |
| |
| |
| |
| Using Single Usability Metric | |
| |
| |
| |
| Usability Scorecards | |
| |
| |
| |
| Comparison to Goals and Expert Performance | |
| |
| |
| |
| Comparison to Goals | |
| |
| |
| |
| Comparison to Expert Performance | |
| |
| |
| |
| Summary | |
| |
| |
| |
| Special Topics | |
| |
| |
| |
| Live Website Data | |
| |
| |
| |
| Basic Web Analytics | |
| |
| |
| |
| Click-Through Rates | |
| |
| |
| |
| Drop-Off Rates | |
| |
| |
| |
| A/B Tests | |
| |
| |
| |
| Card-Sorting Data | |
| |
| |
| |
| Analyses of Open Card-Sort Data | |
| |
| |
| |
| Analyses of Closed Card-Sort Data | |
| |
| |
| |
| Tree Testing | |
| |
| |
| |
| Accessibility Data | |
| |
| |
| |
| Return-On-Investment Data | |
| |
| |
| |
| Summary | |
| |
| |
| |
| Case Studies | |
| |
| |
| |
| Net Promoter Scores and the Value of a Good User Experience | |
| |
| |
| |
| Methods | |
| |
| |
| |
| Results | |
| |
| |
| |
| Prioritizing Investments in Interface Design | |
| |
| |
| |
| Discussion | |
| |
| |
| |
| Conclusion | |
| |
| |
| References | |
| |
| |
| Biographies | |
| |
| |
| |
| Measuring the Effect of Feedback on Fingerprint Capture | |
| |
| |
| |
| Methodology | |
| |
| |
| |
| Discussion | |
| |
| |
| |
| Conclusion | |
| |
| |
| Acknowledgment | |
| |
| |
| References | |
| |
| |
| Biographies | |
| |
| |
| |
| Redesign of a Web Experience Management System | |
| |
| |
| |
| Test Iterations | |
| |
| |
| |
| Data Collection | |
| |
| |
| |
| Workflow | |
| |
| |
| |
| Results | |
| |
| |
| |
| Conclusions | |
| |
| |
| Biographies | |
| |
| |
| |
| Using Metrics to Help Improve a University Prospectus | |
| |
| |
| |
| Example 1: Deciding on Actions after Usability Testing | |
| |
| |
| |
| Example 2: Site-Tracking Data | |
| |
| |
| |
| Example 3: Triangulation for Iteration of Personas | |
| |
| |
| |
| Summary | |
| |
| |
| Acknowledgments | |
| |
| |
| References | |
| |
| |
| Biographies | |
| |
| |
| |
| Measuring Usability Through Biometrics | |
| |
| |
| |
| Background | |
| |
| |
| |
| Methods | |
| |
| |
| |
| Biometric Findings | |
| |
| |
| |
| Qualitative Findings | |
| |
| |
| |
| Conclusions and Practitioner Take-Aways | |
| |
| |
| Acknowledgments | |
| |
| |
| References | |
| |
| |
| Biographies | |
| |
| |
| |
| Ten Keys to Success | |
| |
| |
| |
| Make Data Come Alive | |
| |
| |
| |
| Don't Wait to Be Asked to Measure | |
| |
| |
| |
| Measurement Is Less Expensive Than You Think | |
| |
| |
| |
| Plan Early | |
| |
| |
| |
| Benchmark Your Products | |
| |
| |
| |
| Explore Your Data | |
| |
| |
| |
| Speak the Language of Business | |
| |
| |
| |
| Show Your Confidence | |
| |
| |
| |
| Don't Misuse Metrics | |
| |
| |
| |
| Simplify Your Presentation | |
| |
| |
| References | |
| |
| |
| Index | |