| |
| |
Introduction | |
| |
| |
| |
The Big Picture | |
| |
| |
| |
Software Testing Background | |
| |
| |
Infamous Software Error Case Studies | |
| |
| |
Disney's Lion King, 1994-1995 | |
| |
| |
Intel Pentium Floating-Point Division Bug, 1994 | |
| |
| |
NASA Mars Polar Lander, 1999 | |
| |
| |
Patriot Missile Defense System, 1991 | |
| |
| |
The Y2K (Year 2000) Bug, circa 1974 | |
| |
| |
What Is a Bug? | |
| |
| |
Terms for Software Failures | |
| |
| |
Software Bug: A Formal Definition | |
| |
| |
Why Do Bugs Occur? | |
| |
| |
The Cost of Bugs | |
| |
| |
What Exactly Does a Software Tester Do? | |
| |
| |
What Makes a Good Software Tester? | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
The Software Development Process | |
| |
| |
Product Components | |
| |
| |
What Effort Goes Into a Software Product? | |
| |
| |
What Parts Make Up a Software Product? | |
| |
| |
Software Project Staff | |
| |
| |
Software Development Lifecycle Models | |
| |
| |
Big-Bang Model | |
| |
| |
Code-and-Fix Model | |
| |
| |
Waterfall Model | |
| |
| |
Spiral Model | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
The Realities of Software Testing | |
| |
| |
Testing Axioms | |
| |
| |
It's Impossible to Test a Program Completely | |
| |
| |
Software Testing Is a Risk-Based Exercise | |
| |
| |
Testing Can't Show That Bugs Don't Exist | |
| |
| |
The More Bugs You Find, the More Bugs There Are | |
| |
| |
The Pesticide Paradox | |
| |
| |
Not All the Bugs You Find Will Be Fixed | |
| |
| |
When a Bug's a Bug Is Difficult to Say | |
| |
| |
Product Specifications Are Never Final | |
| |
| |
Software Testers Aren't the Most Popular Members of a Project Team | |
| |
| |
Software Testing Is a Disciplined Technical Profession | |
| |
| |
Software Testing Terms and Definitions | |
| |
| |
Precision and Accuracy | |
| |
| |
Verification and Validation | |
| |
| |
Quality and Reliability | |
| |
| |
Testing and Quality Assurance (QA) | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Testing Fundamentals | |
| |
| |
| |
Examining the Specification | |
| |
| |
Getting Started | |
| |
| |
Black-Box and White-Box Testing | |
| |
| |
Static and Dynamic Testing | |
| |
| |
Static Black-Box Testing: Testing the Specification | |
| |
| |
Performing a High-Level Review of the Specification | |
| |
| |
Pretend to Be the Customer | |
| |
| |
Research Existing Standards and Guidelines | |
| |
| |
Review and Test Similar Software | |
| |
| |
Low-Level Specification Test Techniques | |
| |
| |
Specification Attributes Checklist | |
| |
| |
Specification Terminology Checklist | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Testing the Software with Blinders On | |
| |
| |
Dynamic Black-Box Testing: Testing the Software While Blindfolded | |
| |
| |
Test-to-Pass and Test-to-Fail | |
| |
| |
Equivalence Partitioning | |
| |
| |
Data Testing | |
| |
| |
Boundary Conditions | |
| |
| |
Sub-Boundary Conditions | |
| |
| |
Default, Empty, Blank, Null, Zero, and None | |
| |
| |
Invalid, Wrong, Incorrect, and Garbage Data | |
| |
| |
State Testing | |
| |
| |
Testing the Software's Logic Flow | |
| |
| |
Testing States to Fail | |
| |
| |
Other Black-Box Test Techniques | |
| |
| |
Behave Like a Dumb User | |
| |
| |
Look for Bugs Where You've Already Found Them | |
| |
| |
Follow Experience, Intuition, and Hunches | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Examining the Code | |
| |
| |
Static White-Box Testing: Examining the Design and Code | |
| |
| |
Formal Reviews | |
| |
| |
Peer Reviews | |
| |
| |
Walkthroughts | |
| |
| |
Inspections | |
| |
| |
Coding Standards and Guidelines | |
| |
| |
Examples of Programming Standards and Guidelines | |
| |
| |
Obtaining Standards | |
| |
| |
Generic Code Review Checklist | |
| |
| |
Data Reference Errors | |
| |
| |
Data Declaration Errors | |
| |
| |
Computation Errors | |
| |
| |
Comparison Errors | |
| |
| |
Control Flow Errors | |
| |
| |
Subroutine Parameter Errors | |
| |
| |
Input/Output Errors | |
| |
| |
Other Checks | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Testing the Software with X-Ray Glasses | |
| |
| |
Dynamic White-Box Testing | |
| |
| |
Dynamic White-Box Testing versus Debugging | |
| |
| |
Testing the Pieces | |
| |
| |
Unit and Integration Testing | |
| |
| |
An Example of Module Testing | |
| |
| |
Data Coverage | |
| |
| |
Data Flow | |
| |
| |
Sub-Boundaries | |
| |
| |
Formulas and Equations | |
| |
| |
Error Forcing | |
| |
| |
Code Coverage | |
| |
| |
Program Statement and Line Coverage | |
| |
| |
Branch Coverage | |
| |
| |
Condition Coverage | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Applying Your Testing Skills | |
| |
| |
| |
Configuration Testing | |
| |
| |
An Overview of Configuration Testing | |
| |
| |
Isolating Configuration Bugs | |
| |
| |
Sizing Up the Job | |
| |
| |
Approaching the Task | |
| |
| |
Decide the Types of Hardware You'll Need | |
| |
| |
Decide What Hardware Brands, Models, and Device Drivers Are Available | |
| |
| |
Decide Which Hardware Features, Modes, and Options Are Possible | |
| |
| |
Pare Down the Identified Hardware Configurations to a Manageable Set | |
| |
| |
Identify Your Software's Unique Features That Work with the Hardware Configurations | |
| |
| |
Design the Test Cases to Run on Each Configuration | |
| |
| |
Execute the Tests on Each Configuration | |
| |
| |
Rerun the Tests Until the Results Satisfy Your Team | |
| |
| |
Obtaining the Hardware | |
| |
| |
Identifying Hardware Standards | |
| |
| |
Configuration Testing Other Hardware | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Compatibility Testing | |
| |
| |
Compatibility Testing Overview | |
| |
| |
Platform and Application Versions | |
| |
| |
Backward and Forward Compatibility | |
| |
| |
The Impact of Testing Multiple Versions | |
| |
| |
Standards and Guidelines | |
| |
| |
High-Level Standards and Guidelines | |
| |
| |
Low-Level Standards and Guidelines | |
| |
| |
Data Sharing Compatibility | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Foreign-Language Testing | |
| |
| |
Making the Words and Pictures Make Sense | |
| |
| |
Translation Issues | |
| |
| |
Text Expansion | |
| |
| |
ASCII, DBCS, and Unicode | |
| |
| |
Hot Keys and Shortcuts | |
| |
| |
Extended Characters | |
| |
| |
Computations on Characters | |
| |
| |
Reading Left to Right and Right to Left | |
| |
| |
Text in Graphics | |
| |
| |
Keep the Text out of the Code | |
| |
| |
Localization Issues | |
| |
| |
Content | |
| |
| |
Data Formats | |
| |
| |
Configuration and Compatibility Issues | |
| |
| |
Foreign Platform Configurations | |
| |
| |
Data Compatibility | |
| |
| |
How Much Should You Test? | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Usability Testing | |
| |
| |
User Interface Testing | |
| |
| |
What Makes a Good UI? | |
| |
| |
Follows Standards or Guidelines | |
| |
| |
Intuitive | |
| |
| |
Consistent | |
| |
| |
Flexible | |
| |
| |
Comfortable | |
| |
| |
Correct | |
| |
| |
Useful | |
| |
| |
Testing for the Disabled: Accessibility Testing | |
| |
| |
It's the Law | |
| |
| |
Accessibility Features in Software | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Testing the Documentation | |
| |
| |
Types of Software Documentation | |
| |
| |
The Importance of Documentation Testing | |
| |
| |
What to Look for When Reviewing Documentation | |
| |
| |
The Realities of Documentation Testing | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Web Site Testing | |
| |
| |
Web Page Fundamentals | |
| |
| |
Black-Box Testing | |
| |
| |
Text | |
| |
| |
Hyperlinks | |
| |
| |
Graphics | |
| |
| |
Forms | |
| |
| |
Objects and Other Simple Miscellaneous Functionality | |
| |
| |
Gray-Box Testing | |
| |
| |
White-Box Testing | |
| |
| |
Configuration and Compatibility Testing | |
| |
| |
Usability Testing | |
| |
| |
Introducing Automation | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Supplementing Your Testing | |
| |
| |
| |
Automated Testing and Test Tools | |
| |
| |
The Benefits of Automation and Tools | |
| |
| |
Test Tools | |
| |
| |
Viewers and Monitors | |
| |
| |
Drivers | |
| |
| |
Stubs | |
| |
| |
Stress and Load Tools | |
| |
| |
Interference Injectors and Noise Generators | |
| |
| |
Analysis Tools | |
| |
| |
Software Test Automation | |
| |
| |
Macro Recording and Playback | |
| |
| |
Programmed Macros | |
| |
| |
Fully Programmable Automated Testing Tools | |
| |
| |
Random Testing: Monkeys and Gorillas | |
| |
| |
Dumb Monkeys | |
| |
| |
Semi-Smart Monkeys | |
| |
| |
Smart Monkeys | |
| |
| |
Realities of Using Test Tools and Automation | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Bug Bashes and Beta Testing | |
| |
| |
Only As Far As the Eye Can See | |
| |
| |
Test Sharing | |
| |
| |
Beta Testing | |
| |
| |
Outsourcing Your Testing | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Working with Test Documentation | |
| |
| |
| |
Planning Your Test Effort | |
| |
| |
The Goal of Test Planning | |
| |
| |
Test Planning Topics | |
| |
| |
High-Level Expectations | |
| |
| |
People, Places, and Things | |
| |
| |
Definitions | |
| |
| |
Inter-Group Responsibilities | |
| |
| |
What Will and Won't Be Tested | |
| |
| |
Test Phases | |
| |
| |
Test Strategy | |
| |
| |
Resource Requirements | |
| |
| |
Tester Assignments | |
| |
| |
Test Schedule | |
| |
| |
Test Cases | |
| |
| |
Bug Reporting | |
| |
| |
Metrics and Statistics | |
| |
| |
Risks and Issues | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Writing and Tracking Test Cases | |
| |
| |
The Goals of Test Case Planning | |
| |
| |
Test Case Planning Overview | |
| |
| |
Test Design | |
| |
| |
Test Cases | |
| |
| |
Test Procedures | |
| |
| |
Test Case Organization and Tracking | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Reporting What You Find | |
| |
| |
Getting Your Bugs Fixed | |
| |
| |
Isolating and Reproducing Bugs | |
| |
| |
Not All Bugs Are Created Equal | |
| |
| |
A Bug's Life Cycle | |
| |
| |
Bug-Tracking Systems | |
| |
| |
The Standard: The Test Incident Report | |
| |
| |
Manual Bug Reporting and Tracking | |
| |
| |
Automated Bug Reporting and Tracking | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Measuring Your Success | |
| |
| |
Using the Information in the Bug Tracking Database | |
| |
| |
Metrics That You'll Use in Your Daily Testing | |
| |
| |
Common Project-Level Metrics | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
The Future | |
| |
| |
| |
Software Quality Assurance | |
| |
| |
Quality Is Free | |
| |
| |
Testing and Quality Assurance in the Workplace | |
| |
| |
Software Testing | |
| |
| |
Quality Assurance | |
| |
| |
Other Names for Software Testing Groups | |
| |
| |
Test Management and Organizational Structures | |
| |
| |
Capability Maturity Model (CMM) | |
| |
| |
ISO 9000 | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Your Career as a Software Tester | |
| |
| |
Your Job as a Software Tester | |
| |
| |
Finding a Software Testing Position | |
| |
| |
Gaining Hands-On Experience | |
| |
| |
Formal Training Opportunities | |
| |
| |
Internet Links | |
| |
| |
Professional Organizations | |
| |
| |
Further Reading | |
| |
| |
Summary | |
| |
| |
Quiz | |
| |
| |
| |
Answers to Quiz Questions | |
| |
| |
Index | |