| |
| |
| Introduction | |
| |
| |
| |
| The Big Picture | |
| |
| |
| |
| Software Testing Background | |
| |
| |
| Infamous Software Error Case Studies | |
| |
| |
| Disney's Lion King, 1994-1995 | |
| |
| |
| Intel Pentium Floating-Point Division Bug, 1994 | |
| |
| |
| NASA Mars Polar Lander, 1999 | |
| |
| |
| Patriot Missile Defense System, 1991 | |
| |
| |
| The Y2K (Year 2000) Bug, circa 1974 | |
| |
| |
| What Is a Bug? | |
| |
| |
| Terms for Software Failures | |
| |
| |
| Software Bug: A Formal Definition | |
| |
| |
| Why Do Bugs Occur? | |
| |
| |
| The Cost of Bugs | |
| |
| |
| What Exactly Does a Software Tester Do? | |
| |
| |
| What Makes a Good Software Tester? | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| The Software Development Process | |
| |
| |
| Product Components | |
| |
| |
| What Effort Goes Into a Software Product? | |
| |
| |
| What Parts Make Up a Software Product? | |
| |
| |
| Software Project Staff | |
| |
| |
| Software Development Lifecycle Models | |
| |
| |
| Big-Bang Model | |
| |
| |
| Code-and-Fix Model | |
| |
| |
| Waterfall Model | |
| |
| |
| Spiral Model | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| The Realities of Software Testing | |
| |
| |
| Testing Axioms | |
| |
| |
| It's Impossible to Test a Program Completely | |
| |
| |
| Software Testing Is a Risk-Based Exercise | |
| |
| |
| Testing Can't Show That Bugs Don't Exist | |
| |
| |
| The More Bugs You Find, the More Bugs There Are | |
| |
| |
| The Pesticide Paradox | |
| |
| |
| Not All the Bugs You Find Will Be Fixed | |
| |
| |
| When a Bug's a Bug Is Difficult to Say | |
| |
| |
| Product Specifications Are Never Final | |
| |
| |
| Software Testers Aren't the Most Popular Members of a Project Team | |
| |
| |
| Software Testing Is a Disciplined Technical Profession | |
| |
| |
| Software Testing Terms and Definitions | |
| |
| |
| Precision and Accuracy | |
| |
| |
| Verification and Validation | |
| |
| |
| Quality and Reliability | |
| |
| |
| Testing and Quality Assurance (QA) | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Testing Fundamentals | |
| |
| |
| |
| Examining the Specification | |
| |
| |
| Getting Started | |
| |
| |
| Black-Box and White-Box Testing | |
| |
| |
| Static and Dynamic Testing | |
| |
| |
| Static Black-Box Testing: Testing the Specification | |
| |
| |
| Performing a High-Level Review of the Specification | |
| |
| |
| Pretend to Be the Customer | |
| |
| |
| Research Existing Standards and Guidelines | |
| |
| |
| Review and Test Similar Software | |
| |
| |
| Low-Level Specification Test Techniques | |
| |
| |
| Specification Attributes Checklist | |
| |
| |
| Specification Terminology Checklist | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Testing the Software with Blinders On | |
| |
| |
| Dynamic Black-Box Testing: Testing the Software While Blindfolded | |
| |
| |
| Test-to-Pass and Test-to-Fail | |
| |
| |
| Equivalence Partitioning | |
| |
| |
| Data Testing | |
| |
| |
| Boundary Conditions | |
| |
| |
| Sub-Boundary Conditions | |
| |
| |
| Default, Empty, Blank, Null, Zero, and None | |
| |
| |
| Invalid, Wrong, Incorrect, and Garbage Data | |
| |
| |
| State Testing | |
| |
| |
| Testing the Software's Logic Flow | |
| |
| |
| Testing States to Fail | |
| |
| |
| Other Black-Box Test Techniques | |
| |
| |
| Behave Like a Dumb User | |
| |
| |
| Look for Bugs Where You've Already Found Them | |
| |
| |
| Follow Experience, Intuition, and Hunches | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Examining the Code | |
| |
| |
| Static White-Box Testing: Examining the Design and Code | |
| |
| |
| Formal Reviews | |
| |
| |
| Peer Reviews | |
| |
| |
| Walkthroughts | |
| |
| |
| Inspections | |
| |
| |
| Coding Standards and Guidelines | |
| |
| |
| Examples of Programming Standards and Guidelines | |
| |
| |
| Obtaining Standards | |
| |
| |
| Generic Code Review Checklist | |
| |
| |
| Data Reference Errors | |
| |
| |
| Data Declaration Errors | |
| |
| |
| Computation Errors | |
| |
| |
| Comparison Errors | |
| |
| |
| Control Flow Errors | |
| |
| |
| Subroutine Parameter Errors | |
| |
| |
| Input/Output Errors | |
| |
| |
| Other Checks | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Testing the Software with X-Ray Glasses | |
| |
| |
| Dynamic White-Box Testing | |
| |
| |
| Dynamic White-Box Testing versus Debugging | |
| |
| |
| Testing the Pieces | |
| |
| |
| Unit and Integration Testing | |
| |
| |
| An Example of Module Testing | |
| |
| |
| Data Coverage | |
| |
| |
| Data Flow | |
| |
| |
| Sub-Boundaries | |
| |
| |
| Formulas and Equations | |
| |
| |
| Error Forcing | |
| |
| |
| Code Coverage | |
| |
| |
| Program Statement and Line Coverage | |
| |
| |
| Branch Coverage | |
| |
| |
| Condition Coverage | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Applying Your Testing Skills | |
| |
| |
| |
| Configuration Testing | |
| |
| |
| An Overview of Configuration Testing | |
| |
| |
| Isolating Configuration Bugs | |
| |
| |
| Sizing Up the Job | |
| |
| |
| Approaching the Task | |
| |
| |
| Decide the Types of Hardware You'll Need | |
| |
| |
| Decide What Hardware Brands, Models, and Device Drivers Are Available | |
| |
| |
| Decide Which Hardware Features, Modes, and Options Are Possible | |
| |
| |
| Pare Down the Identified Hardware Configurations to a Manageable Set | |
| |
| |
| Identify Your Software's Unique Features That Work with the Hardware Configurations | |
| |
| |
| Design the Test Cases to Run on Each Configuration | |
| |
| |
| Execute the Tests on Each Configuration | |
| |
| |
| Rerun the Tests Until the Results Satisfy Your Team | |
| |
| |
| Obtaining the Hardware | |
| |
| |
| Identifying Hardware Standards | |
| |
| |
| Configuration Testing Other Hardware | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Compatibility Testing | |
| |
| |
| Compatibility Testing Overview | |
| |
| |
| Platform and Application Versions | |
| |
| |
| Backward and Forward Compatibility | |
| |
| |
| The Impact of Testing Multiple Versions | |
| |
| |
| Standards and Guidelines | |
| |
| |
| High-Level Standards and Guidelines | |
| |
| |
| Low-Level Standards and Guidelines | |
| |
| |
| Data Sharing Compatibility | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Foreign-Language Testing | |
| |
| |
| Making the Words and Pictures Make Sense | |
| |
| |
| Translation Issues | |
| |
| |
| Text Expansion | |
| |
| |
| ASCII, DBCS, and Unicode | |
| |
| |
| Hot Keys and Shortcuts | |
| |
| |
| Extended Characters | |
| |
| |
| Computations on Characters | |
| |
| |
| Reading Left to Right and Right to Left | |
| |
| |
| Text in Graphics | |
| |
| |
| Keep the Text out of the Code | |
| |
| |
| Localization Issues | |
| |
| |
| Content | |
| |
| |
| Data Formats | |
| |
| |
| Configuration and Compatibility Issues | |
| |
| |
| Foreign Platform Configurations | |
| |
| |
| Data Compatibility | |
| |
| |
| How Much Should You Test? | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Usability Testing | |
| |
| |
| User Interface Testing | |
| |
| |
| What Makes a Good UI? | |
| |
| |
| Follows Standards or Guidelines | |
| |
| |
| Intuitive | |
| |
| |
| Consistent | |
| |
| |
| Flexible | |
| |
| |
| Comfortable | |
| |
| |
| Correct | |
| |
| |
| Useful | |
| |
| |
| Testing for the Disabled: Accessibility Testing | |
| |
| |
| It's the Law | |
| |
| |
| Accessibility Features in Software | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Testing the Documentation | |
| |
| |
| Types of Software Documentation | |
| |
| |
| The Importance of Documentation Testing | |
| |
| |
| What to Look for When Reviewing Documentation | |
| |
| |
| The Realities of Documentation Testing | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Web Site Testing | |
| |
| |
| Web Page Fundamentals | |
| |
| |
| Black-Box Testing | |
| |
| |
| Text | |
| |
| |
| Hyperlinks | |
| |
| |
| Graphics | |
| |
| |
| Forms | |
| |
| |
| Objects and Other Simple Miscellaneous Functionality | |
| |
| |
| Gray-Box Testing | |
| |
| |
| White-Box Testing | |
| |
| |
| Configuration and Compatibility Testing | |
| |
| |
| Usability Testing | |
| |
| |
| Introducing Automation | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Supplementing Your Testing | |
| |
| |
| |
| Automated Testing and Test Tools | |
| |
| |
| The Benefits of Automation and Tools | |
| |
| |
| Test Tools | |
| |
| |
| Viewers and Monitors | |
| |
| |
| Drivers | |
| |
| |
| Stubs | |
| |
| |
| Stress and Load Tools | |
| |
| |
| Interference Injectors and Noise Generators | |
| |
| |
| Analysis Tools | |
| |
| |
| Software Test Automation | |
| |
| |
| Macro Recording and Playback | |
| |
| |
| Programmed Macros | |
| |
| |
| Fully Programmable Automated Testing Tools | |
| |
| |
| Random Testing: Monkeys and Gorillas | |
| |
| |
| Dumb Monkeys | |
| |
| |
| Semi-Smart Monkeys | |
| |
| |
| Smart Monkeys | |
| |
| |
| Realities of Using Test Tools and Automation | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Bug Bashes and Beta Testing | |
| |
| |
| Only As Far As the Eye Can See | |
| |
| |
| Test Sharing | |
| |
| |
| Beta Testing | |
| |
| |
| Outsourcing Your Testing | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Working with Test Documentation | |
| |
| |
| |
| Planning Your Test Effort | |
| |
| |
| The Goal of Test Planning | |
| |
| |
| Test Planning Topics | |
| |
| |
| High-Level Expectations | |
| |
| |
| People, Places, and Things | |
| |
| |
| Definitions | |
| |
| |
| Inter-Group Responsibilities | |
| |
| |
| What Will and Won't Be Tested | |
| |
| |
| Test Phases | |
| |
| |
| Test Strategy | |
| |
| |
| Resource Requirements | |
| |
| |
| Tester Assignments | |
| |
| |
| Test Schedule | |
| |
| |
| Test Cases | |
| |
| |
| Bug Reporting | |
| |
| |
| Metrics and Statistics | |
| |
| |
| Risks and Issues | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Writing and Tracking Test Cases | |
| |
| |
| The Goals of Test Case Planning | |
| |
| |
| Test Case Planning Overview | |
| |
| |
| Test Design | |
| |
| |
| Test Cases | |
| |
| |
| Test Procedures | |
| |
| |
| Test Case Organization and Tracking | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Reporting What You Find | |
| |
| |
| Getting Your Bugs Fixed | |
| |
| |
| Isolating and Reproducing Bugs | |
| |
| |
| Not All Bugs Are Created Equal | |
| |
| |
| A Bug's Life Cycle | |
| |
| |
| Bug-Tracking Systems | |
| |
| |
| The Standard: The Test Incident Report | |
| |
| |
| Manual Bug Reporting and Tracking | |
| |
| |
| Automated Bug Reporting and Tracking | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Measuring Your Success | |
| |
| |
| Using the Information in the Bug Tracking Database | |
| |
| |
| Metrics That You'll Use in Your Daily Testing | |
| |
| |
| Common Project-Level Metrics | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| The Future | |
| |
| |
| |
| Software Quality Assurance | |
| |
| |
| Quality Is Free | |
| |
| |
| Testing and Quality Assurance in the Workplace | |
| |
| |
| Software Testing | |
| |
| |
| Quality Assurance | |
| |
| |
| Other Names for Software Testing Groups | |
| |
| |
| Test Management and Organizational Structures | |
| |
| |
| Capability Maturity Model (CMM) | |
| |
| |
| ISO 9000 | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Your Career as a Software Tester | |
| |
| |
| Your Job as a Software Tester | |
| |
| |
| Finding a Software Testing Position | |
| |
| |
| Gaining Hands-On Experience | |
| |
| |
| Formal Training Opportunities | |
| |
| |
| Internet Links | |
| |
| |
| Professional Organizations | |
| |
| |
| Further Reading | |
| |
| |
| Summary | |
| |
| |
| Quiz | |
| |
| |
| |
| Answers to Quiz Questions | |
| |
| |
| Index | |