Rights Contact Login For More Details
- Wiley
More About This Title Software Testing: Concepts and Operations
English
Explores and identifies the main issues, concepts, principles and evolution of software testing, including software quality engineering and testing concepts, test data generation, test deployment analysis, and software test management
This book examines the principles, concepts, and processes that are fundamental to the software testing function. This book is divided into five broad parts. Part I introduces software testing in the broader context of software engineering and explores the qualities that testing aims to achieve or ascertain, as well as the lifecycle of software testing. Part II covers mathematical foundations of software testing, which include software specification, program correctness and verification, concepts of software dependability, and a software testing taxonomy. Part III discusses test data generation, specifically, functional criteria and structural criteria. Test oracle design, test driver design, and test outcome analysis is covered in Part IV. Finally, Part V surveys managerial aspects of software testing, including software metrics, software testing tools, and software product line testing.
- Presents software testing, not as an isolated technique, but as part of an integrated discipline of software verification and validation
- Proposes program testing and program correctness verification within the same mathematical model, making it possible to deploy the two techniques in concert, by virtue of the law of diminishing returns
- Defines the concept of a software fault, and the related concept of relative correctness, and shows how relative correctness can be used to characterize monotonic fault removal
- Presents the activity of software testing as a goal oriented activity, and explores how the conduct of the test depends on the selected goal
- Covers all phases of the software testing lifecycle, including test data generation, test oracle design, test driver design, and test outcome analysis
English
Ali Mili is a Professor at New Jersey Institute of Technology. His research is focused on software engineering, including technical and organizational aspects, and on software engineering education.
English
PREFACE XIV
ACKNOWLEDGMENTS XVIII
PART I INTRODUCTION TO SOFTWARE TESTING 1
1 Software Engineering: A Discipline Like No Other 3
1.1 A Young, Restless Discipline 3
1.2 An Industry Under Stress 5
1.3 Large, Complex Products 5
1.4 Expensive Products 7
1.5 Absence of Reuse Practice 9
1.6 Fault-Prone Designs 9
1.7 Paradoxical Economics 10
1.7.1 A Labor-Intensive Industry 10
1.7.2 Absence of Automation 11
1.7.3 Limited Quality Control 11
1.7.4 Unbalanced Lifecycle Costs 12
1.7.5 Unbalanced Maintenance Costs 12
1.8 Chapter Summary 13
1.9 Bibliographic Notes 13
2 Software Quality Attributes 14
2.1 Functional Attributes 15
2.1.1 Boolean Attributes 15
2.1.2 Statistical Attributes 15
2.2 Operational Attributes 17
2.3 Usability Attributes 18
2.4 Business Attributes 19
2.5 Structural Attributes 20
2.6 Chapter Summary 21
2.7 Exercises 21
2.8 Bibliographic Notes 22
3 A Software Testing Lifecycle 23
3.1 A Software Engineering Lifecycle 23
3.2 A Software Testing Lifecycle 28
3.3 The V-Model of Software Testing 31
3.4 Chapter Summary 34
3.5 Bibliographic Notes 34
PART II FOUNDATIONS OF SOFTWARE TESTING 35
4 Software Specifications 37
4.1 Principles of Sound Specification 38
4.1.1 A Discipline of Specification 38
4.2 Relational Mathematics 39
4.2.1 Sets and Relations 39
4.2.2 Operations on Relations 39
4.2.3 Properties of Relations 41
4.3 Simple Input Output Programs 42
4.4 Representing Specifications 42
4.5 Ordering Specifications 46
4.6 Specification Generation 48
4.7 Specification Validation 53
4.7.1 Specification Generation Phase 56
4.7.2 Specification Validation Phase 57
4.8 Reliability Versus Safety 60
4.9 State-based Systems 61
4.9.1 A Relational Model 62
4.10 Axiomatic Representation 64
4.10.1 Specification of a Stack 65
4.10.2 Specification of a Queue 66
4.10.3 Specification of a Set 68
4.11 Specification Validation 69
4.12 Chapter Summary 71
4.13 Exercises 72
4.14 Problems 76
4.15 Bibliographic Notes 77
5 Program Correctness and Verification 79
5.1 Correctness: A Definition 80
5.2 Correctness: Propositions 83
5.2.1 Correctness and Refinement 83
5.2.2 Set Theoretic Characterizations 85
5.2.3 Illustrations 86
5.3 Verification 88
5.3.1 Sample Formulas 89
5.3.2 An Inference System 91
5.3.3 Illustrative Examples 94
5.4 Chapter Summary 98
5.5 Exercises 99
5.6 Problems 100
5.7 Bibliographic Notes 100
6 Failures, Errors, and Faults 101
6.1 Failure, Error, and Fault 101
6.2 Faults and Relative Correctness 103
6.2.1 Fault, an Evasive Concept 103
6.2.2 Relative Correctness 104
6.3 Contingent Faults and Definite Faults 107
6.3.1 Contingent Faults 107
6.3.2 Monotonic Fault Removal 109
6.3.3 A Framework for Monotonic Fault Removal 114
6.3.4 Definite Faults 114
6.4 Fault Management 116
6.4.1 Lines of Defense 116
6.4.2 Hybrid Validation 118
6.5 Chapter Summary 121
6.6 Exercises 122
6.7 Problems 123
6.8 Bibliographic Notes 124
7 A Software Testing Taxonomy 125
7.1 The Trouble with Hyphenated Testing 125
7.2 A Classification Scheme 126
7.2.1 Primary Attributes 127
7.2.2 Secondary Attributes 131
7.3 Testing Taxonomy 136
7.3.1 Unit-Level Testing 136
7.3.2 System-Level Testing 138
7.4 Exercises 139
7.5 Bibliographic Notes 140
PART III TEST DATA GENERATION 141
8 Test Generation Concepts 143
8.1 Test Generation and Target Attributes 143
8.2 Test Outcomes 146
8.3 Test Generation Requirements 148
8.4 Test Generation Criteria 152
8.5 Empirical Adequacy Assessment 155
8.6 Chapter Summary 160
8.7 Exercises 161
8.8 Bibliographic Notes 162
8.9 Appendix: Mutation Program 163
9 Functional Criteria 165
9.1 Domain Partitioning 165
9.2 Test Data Generation from Tabular Expressions 171
9.3 Test Generation for State Based Systems 176
9.4 Random Test Data Generation 184
9.5 Tourism as a Metaphor for Test Data Selection 188
9.6 Chapter Summary 190
9.7 Exercises 190
9.8 Bibliographic Notes 192
10 Structural Criteria 193
10.1 Paths and Path Conditions 194
10.1.1 Execution Paths 194
10.1.2 Path Functions 196
10.1.3 Path Conditions 201
10.2 Control Flow Coverage 202
10.2.1 Statement Coverage 202
10.2.2 Branch Coverage 204
10.2.3 Condition Coverage 207
10.2.4 Path Coverage 209
10.3 Data Flow Coverage 214
10.3.1 Definitions and Uses 214
10.3.2 Test Generation Criteria 217
10.3.3 A Hierarchy of Criteria 220
10.4 Fault-Based Test Generation 220
10.4.1 Sensitizing Faults 221
10.4.2 Selecting Input Data for Fault Sensitization 225
10.4.3 Selecting Input Data for Error Propagation 227
10.5 Chapter Summary 228
10.6 Exercises 229
10.7 Bibliographic Notes 232
PART IV TEST DEPLOYMENT AND ANALYSIS 233
11 Test Oracle Design 235
11.1 Dilemmas of Oracle Design 235
11.2 From Specifications to Oracles 238
11.3 Oracles for State-Based Products 242
11.3.1 From Axioms to Oracles 243
11.3.2 From Rules to Oracles 244
11.4 Chapter Summary 250
11.5 Exercises 251
12 Test Driver Design 254
12.1 Selecting a Specification 254
12.2 Selecting a Process 256
12.3 Selecting a Specification Model 258
12.3.1 Random Test Generation 258
12.3.2 Pre-Generated Test Data 264
12.3.3 Faults and Fault Detection 267
12.4 Testing by Symbolic Execution 270
12.5 Chapter Summary 275
12.6 Exercises 276
12.7 Bibliographic Notes 280
13 Test Outcome Analysis 281
13.1 Logical Claims 282
13.1.1 Concrete Testing 282
13.1.2 Symbolic Testing 283
13.1.3 Concolic Testing 284
13.2 Stochastic Claims: Fault Density 285
13.3 Stochastic Claims: Failure Probability 288
13.3.1 Faults are not Created Equal 288
13.3.2 Defining/Quantifying Reliability 290
13.3.3 Modeling Software Reliability 292
13.3.4 Certification Testing 295
13.3.5 Reliability Estimation and Reliability Improvement 296
13.3.6 Reliability Standards 300
13.3.7 Reliability as an Economic Function 301
13.4 Chapter Summary 308
13.5 Exercises 309
13.6 Problems 311
13.7 Bibliographic Notes 311
PART V MANAGEMENT OF SOFTWARE TESTING 313
14 Metrics for Software Testing 315
14.1 Fault Proneness 316
14.1.1 Cyclomatic Complexity 317
14.1.2 Volume 318
14.2 Fault Detectability 319
14.3 Error Detectability 322
14.4 Error Maskability 325
14.5 Failure Avoidance 326
14.6 Failure Tolerance 328
14.7 An Illustrative Example 329
14.7.1 Cyclomatic Complexity 329
14.7.2 Volume 330
14.7.3 State Redundancy 330
14.7.4 Functional Redundancy 330
14.7.5 Non-injectivity 331
14.7.6 Non-determinacy 331
14.7.7 Summary 332
14.8 Chapter Summary 332
14.9 Exercises 333
14.10 Bibliographic Notes 334
15 Software Testing Tools 335
15.1 A Classification Scheme 335
15.2 Scripting Tools 336
15.2.1 CppTest 336
15.2.2 SilkTest 337
15.3 Record-and-Replay Tools 338
15.3.1 TestComplete 338
15.3.2 Selenium IDE 339
15.4 Performance-Testing Tools 340
15.4.1 LoadRunner 340
15.4.2 Grinder 341
15.4.3 QF-Test 342
15.4.4 Appvance PerformanceCloud 342
15.4.5 JMeter 343
15.5 Oracle Design Tools 344
15.5.1 JUnit 344
15.5.2 TestNG 345
15.6 Exception Discovery 345
15.6.1 Rational Purify 345
15.6.2 Astree 346
15.7 Collaborative Tools 347
15.7.1 FitNesse 347
15.8 Chapter Summary 347
16 Testing Product Lines 349
16.1 PLE: A Streamlined reuse model 349
16.2 Testing Issues 353
16.3 Testing Approaches 355
16.4 Illustration 356
16.4.1 Domain Analysis 356
16.4.2 Domain Modeling 358
16.4.3 A Reference Architecture 361
16.4.4 Domain Implementation 362
16.4.5 Testing at Domain Engineering 367
16.4.6 Testing at Application Engineering 371
16.5 Chapter Summary 374
16.6 Exercises 374
16.7 Problems 374
16.8 Bibliographic References 375
BIBLIOGRAPHY 376
INDEX 000
