| |
| |
Preface | |
| |
| |
Acknowledgments | |
| |
| |
| |
Preliminaries | |
| |
| |
| |
Basics of Software Testing | |
| |
| |
| |
Humans, errors, and testing | |
| |
| |
| |
Errors, faults, and failures | |
| |
| |
| |
Test automation | |
| |
| |
| |
Developer and tester as two roles | |
| |
| |
| |
Software quality | |
| |
| |
| |
Quality attributes | |
| |
| |
| |
Reliability | |
| |
| |
| |
Requirements, behavior, and correctiness | |
| |
| |
| |
Input domain and program correctness | |
| |
| |
| |
Valid and invalid inputs | |
| |
| |
| |
Correctness versus reliability | |
| |
| |
| |
Correctness | |
| |
| |
| |
Reliability | |
| |
| |
| |
Program use and the operational profile | |
| |
| |
| |
Testing and debugging | |
| |
| |
| |
Preparing a test plan | |
| |
| |
| |
Constructing test data | |
| |
| |
| |
Executing the program | |
| |
| |
| |
Specifying program behavior | |
| |
| |
| |
Assessing the correctness of program behavior | |
| |
| |
| |
Construction of oracles | |
| |
| |
| |
Test metrics | |
| |
| |
| |
Organizational metrics | |
| |
| |
| |
Project metrics | |
| |
| |
| |
Process metrics | |
| |
| |
| |
Product metrics: Generic | |
| |
| |
| |
Product metrics: OO software | |
| |
| |
| |
Progress monitoring and trends | |
| |
| |
| |
Static and dynamic metrics | |
| |
| |
| |
Testability | |
| |
| |
| |
Software and hardware testing | |
| |
| |
| |
Testing and verification | |
| |
| |
| |
Defect management | |
| |
| |
| |
Execution history | |
| |
| |
| |
Test-generation strategies | |
| |
| |
| |
Static testing | |
| |
| |
| |
Walkthroughs | |
| |
| |
| |
Inspections | |
| |
| |
| |
Use of static code analysis tools in static testing | |
| |
| |
| |
Software complexity and static testing | |
| |
| |
| |
Model-based testing and model checking | |
| |
| |
| |
Control-flow graph | |
| |
| |
| |
Basic block | |
| |
| |
| |
Flow graph: Definition and pictorial representation | |
| |
| |
| |
Path | |
| |
| |
| |
Dominators and postdominators | |
| |
| |
| |
Program-dependence graph | |
| |
| |
| |
Data dependence | |
| |
| |
| |
Control dependence | |
| |
| |
| |
Strings, languages, and regular expressions | |
| |
| |
| |
Types of testing | |
| |
| |
| |
Classifier C1: Source of test generation | |
| |
| |
| |
Classifier C2: Life cycle phase | |
| |
| |
| |
Classifier C3: Goal-directed testing | |
| |
| |
| |
Classifier C4: Artifact under test | |
| |
| |
| |
Classifier C5: Test process models | |
| |
| |
| |
The saturation effect | |
| |
| |
| |
Confidence and true reliability | |
| |
| |
| |
Saturation region | |
| |
| |
| |
False sense of confidence | |
| |
| |
| |
Reducing [Delta] | |
| |
| |
| |
Impact on test process | |
| |
| |
Summary | |
| |
| |
Bibliographic Notes | |
| |
| |
Exercises | |
| |
| |
| |
Test Generation | |
| |
| |
| |
Test Generation from Requirements | |
| |
| |
| |
Introduction | |
| |
| |
| |
The test-selection problem | |
| |
| |
| |
Equivalence partitioning | |
| |
| |
| |
Faults targeted | |
| |
| |
| |
Relations and equivalence partitioning | |
| |
| |
| |
Equivalence classes for variables | |
| |
| |
| |
Unidimensional versus multidimensional partitioning | |
| |
| |
| |
A systematic procedure for equivalence partitioning | |
| |
| |
| |
Test selection based on equivalence classes | |
| |
| |
| |
GUI design and equivalence classes | |
| |
| |
| |
Boundary-value analysis | |
| |
| |
| |
Category-partition method | |
| |
| |
| |
Steps in the category-partition method | |
| |
| |
| |
Cause-effect graphing | |
| |
| |
| |
Notation used in cause-effect graphing | |
| |
| |
| |
Creating cause-effect graphs | |
| |
| |
| |
Decision table from cause-effect graph | |
| |
| |
| |
Heuristics to avoid combinatorial explosion | |
| |
| |
| |
Test generation from a decision table | |
| |
| |
| |
Test generation from predicates | |
| |
| |
| |
Predicates and boolean expressions | |
| |
| |
| |
Fault model for predicate testing | |
| |
| |
| |
Predicate constraints | |
| |
| |
| |
Predicate-testing criteria | |
| |
| |
| |
Generating BOR-, BRO-, and BRE-adequate tests | |
| |
| |
| |
Cause-effect graphs and predicate testing | |
| |
| |
| |
Fault propagation | |
| |
| |
| |
Predicate testing in practice | |
| |
| |
Summary | |
| |
| |
Bibliographic Notes | |
| |
| |
Exercises | |
| |
| |
| |
Test Generation from Finite-State Models | |
| |
| |
| |
Software design and testing | |
| |
| |
| |
Finite-state machines | |
| |
| |
| |
Excitation using an input sequence | |
| |
| |
| |
Tabular representation | |
| |
| |
| |
Properties of FSM | |
| |
| |
| |
Conformance testing | |
| |
| |
| |
Reset inputs | |
| |
| |
| |
The testing problem | |
| |
| |
| |
A fault model | |
| |
| |
| |
Mutants of FSMs | |
| |
| |
| |
Fault coverage | |
| |
| |
| |
Characterization set | |
| |
| |
| |
Construction of the k-equivalence partitions | |
| |
| |
| |
Deriving the characterization set | |
| |
| |
| |
Identification sets | |
| |
| |
| |
The w-method | |
| |
| |
| |
Assumptions | |
| |
| |
| |
Maximum number of states | |
| |
| |
| |
Computation of the transition cover set | |
| |
| |
| |
Constructing Z | |
| |
| |
| |
Deriving a test set | |
| |
| |
| |
Testing using the W-method | |
| |
| |
| |
The error-detection process | |
| |
| |
| |
The partial w-method | |
| |
| |
| |
Testing using the Wp-method for m = n | |
| |
| |
| |
Testing using the Wp-method for m > n | |
| |
| |
| |
The UIO-sequence method | |
| |
| |
| |
Assumptions | |
| |
| |
| |
UIO sequences | |
| |
| |
| |
Core and noncore behavior | |
| |
| |
| |
Generation of UIO sequences | |
| |
| |
| |
Distinguishing signatures | |
| |
| |
| |
Test generation | |
| |
| |
| |
Test optimization | |
| |
| |
| |
Fault detection | |
| |
| |
| |
Automata theoretic versus control-flow-based techniques | |
| |
| |
| |
n-switch-cover | |
| |
| |
| |
Comparing automata-theoretic methods | |
| |
| |
Summary | |
| |
| |
Bibliographic Notes | |
| |
| |
Exercises | |
| |
| |
| |
Test Generation from Combinatorial Designs | |
| |
| |
| |
Combinatorial designs | |
| |
| |
| |
Test configuration and test set | |
| |
| |
| |
Modeling the input and configuration spaces | |
| |
| |
| |
A combinatorial test-design process | |
| |
| |
| |
Fault model | |
| |
| |
| |
Fault vectors | |
| |
| |
| |
Latin squares | |
| |
| |
| |
Mutually orthogonal latin squares | |
| |
| |
| |
Pairwise design: binary factors | |
| |
| |
| |
Pairwise design: multivalued factors | |
| |
| |
| |
Shortcomings of using MOLS for test design | |
| |
| |
| |
Orthogonal arrays | |
| |
| |
| |
Mixed-level orthogonal arrays | |
| |
| |
| |
Covering and mixed-level covering arrays | |
| |
| |
| |
Covering arrays | |
| |
| |
| |
Mixed-level covering arrays | |
| |
| |
| |
Arrays of strength >2 | |
| |
| |
| |
Generating covering arrays | |
| |
| |
Summary | |
| |
| |
Bibliographic Notes | |
| |
| |
Exercises | |
| |
| |
| |
Test Selection, Minimizations, and Priaritization for Regression Testing | |
| |
| |
| |
What is regression testing? | |
| |
| |
| |
Regression-test process | |
| |
| |
| |
Test revalidation, selection, minimization, and prioritization | |
| |
| |
| |
Test setup | |
| |
| |
| |
Test sequencing | |
| |
| |
| |
Test execution | |
| |
| |
| |
Output comparison | |
| |
| |
| |
RTS: the problem | |
| |
| |
| |
Selecting regression tests | |
| |
| |
| |
Test all | |
| |
| |
| |
Random selection | |
| |
| |
| |
Selecting modification-traversing tests | |
| |
| |
| |
Test minimization | |
| |
| |
| |
Test prioritization | |
| |
| |
| |
Test selection using execution trace | |
| |
| |
| |
Obtaining the execution trace | |
| |
| |
| |
Selecting regression tests | |
| |
| |
| |
Handling function calls | |
| |
| |
| |
Handling changes in declarations | |
| |
| |
| |
Test selection using dynamic slicing | |
| |
| |
| |
Dynamic slicing | |
| |
| |
| |
Computation of dynamic slices | |
| |
| |
| |
Selecting tests | |
| |
| |
| |
Potential dependence | |
| |
| |
| |
Computing the relevant slice | |
| |
| |
| |
Addition and deletion of statements | |
| |
| |
| |
Identifying variables for slicing | |
| |
| |
| |
Reduced dynamic-dependence graph | |
| |
| |
| |
Scalability of test-selection algorithms | |
| |
| |
| |
Test minimization | |
| |
| |
| |
The set-cover problem | |
| |
| |
| |
A procedure for test minimization | |
| |
| |
| |
Test prioritization | |
| |
| |
| |
Tools for regression testing | |
| |
| |
Summary | |
| |
| |
Bibliographic Notes | |
| |
| |
Exercises | |
| |
| |
| |
Test Adequacy Assessment and Enhancement | |
| |
| |
| |
Test Adequacy: Assessment using Control Flow and Data Flow | |
| |
| |
| |
Test adequacy: basics | |
| |
| |
| |
What is test adequacy? | |
| |
| |
| |
Measurement of test adequacy | |
| |
| |
| |
Test enhancement using measurements of adequacy | |
| |
| |
| |
Infeasibility and test adequacy | |
| |
| |
| |
Error detection and test enhancement | |
| |
| |
| |
Single and multiple executions | |
| |
| |
| |
Adequacy criteria based on control flow | |
| |
| |
| |
Statement and block coverage | |
| |
| |
| |
Conditions and decisions | |
| |
| |
| |
Decision coverage | |
| |
| |
| |
Condition coverage | |
| |
| |
| |
Condition/decision coverage | |
| |
| |
| |
Multiple condition coverage | |
| |
| |
| |
Linear code sequence and jump (LCSAJ) coverage | |
| |
| |
| |
Modified condition/decision coverage | |
| |
| |
| |
MC/DC-adequate tests for compound conditions | |
| |
| |
| |
Definition of MC/DC coverage | |
| |
| |
| |
Minimal MC/DC tests | |
| |
| |
| |
Error detection and MC/DC adequacy | |
| |
| |
| |
Short-circuit evaluation and infeasibility | |
| |
| |
| |
Tracing test cases to requirements | |
| |
| |
| |
Data-flow concepts | |
| |
| |
| |
Definitions and uses | |
| |
| |
| |
c-use and p-use | |
| |
| |
| |
Global and local definitions and uses | |
| |
| |
| |
Data-flow graph | |
| |
| |
| |
Def-clear paths | |
| |
| |
| |
Def-use pairs | |
| |
| |
| |
Def-use chains | |
| |
| |
| |
A little optimization | |
| |
| |
| |
Data contexts and ordered data contexts | |
| |
| |
| |
Adequacy criteria based on data flow | |
| |
| |
| |
c-use coverage | |
| |
| |
| |
p-use coverage | |
| |
| |
| |
all-uses coverage | |
| |
| |
| |
k-dr chain coverage | |
| |
| |
| |
Using the k-dr chain coverage | |
| |
| |
| |
Infeasible c-uses and p-uses | |
| |
| |
| |
Context coverage | |
| |
| |
| |
Control flow versus data flow | |
| |
| |
| |
The subsumes relation | |
| |
| |
| |
Structural and functional testing | |
| |
| |
| |
Scalability of coverage measurement | |
| |
| |
Summary | |
| |
| |
Bibliographic Notes | |
| |
| |
Exercises | |
| |
| |
| |
Test-Adequacy Assessment Using Program Mutation | |
| |
| |
| |
Introduction | |
| |
| |
| |
Mutation and mutants | |
| |
| |
| |
First-order and higher-order mutants | |
| |
| |
| |
Syntax and semantics of mutants | |
| |
| |
| |
Strong and weak mutations | |
| |
| |
| |
Why mutate? | |
| |
| |
| |
Test assessment using mutation | |
| |
| |
| |
A procedure for test-adequacy assessment | |
| |
| |
| |
Alternate procedures for test-adequacy assessment | |
| |
| |
| |
Distinguished versus killed mutants | |
| |
| |
| |
Conditions for distinguishing a mutant | |
| |
| |
| |
Mutation operators | |
| |
| |
| |
Operator types | |
| |
| |
| |
Language dependence of mutation operators | |
| |
| |
| |
Design of mutation operators | |
| |
| |
| |
Goodness criteria for mutation operators | |
| |
| |
| |
Guidelines | |
| |
| |
| |
Founding principles of mutation testing | |
| |
| |
| |
The competent programmer hypothesis | |
| |
| |
| |
The coupling effect | |
| |
| |
| |
Equivalent mutants | |
| |
| |
| |
Fault detection using mutation | |
| |
| |
| |
Types of mutants | |
| |
| |
| |
Mutation operators for C | |
| |
| |
| |
What is not mutated? | |
| |
| |
| |
Linearization | |
| |
| |
| |
Execution sequence | |
| |
| |
| |
Effect of an execution sequence | |
| |
| |
| |
Global and local identifier sets | |
| |
| |
| |
Global and local reference sets | |
| |
| |
| |
Mutating program constants | |
| |
| |
| |
Mutating operators | |
| |
| |
| |
Mutating statements | |
| |
| |
| |
Mutating program variables | |
| |
| |
| |
Mutation operators for java | |
| |
| |
| |
Traditional mutation operators | |
| |
| |
| |
Inheritence | |
| |
| |
| |
Polymorphism and dynamic binding | |
| |
| |
| |
Method overloading | |
| |
| |
| |
Java-specific mutation operators | |
| |
| |
| |
Mutation operators for fortran 77, C, and java: a comparison | |
| |
| |
| |
Tools for mutation testing | |
| |
| |
| |
Mutation testing within budget | |
| |
| |
| |
Prioritizing functions to be mutated | |
| |
| |
| |
Selecting a subset of mutation operators | |
| |
| |
Summary | |
| |
| |
Bibliographic Notes | |
| |
| |
Exercises | |
| |
| |
References | |
| |
| |
Subject Index | |
| |
| |
Name Index | |