略过1-3章,直接从第四章开始读。
Chapter4:A Practical Approach to Test Case Design
Practicing good software design and test design
Test design has many parallels with good software design and with good design in general. Test design requires planning and problem solving to determine which kinds of tests to run and which types of testing will be most effective at verifying functionality and confirming that failure path are handled gracefully. One of the most important aspects of test design is anticipating customer needs and expectations and then creating tests that address these needs appropriately .Good test design often starts with a review, or critique, of the software design.
测试设计需要做的是找出需要做哪些测试以及哪些测试能更有效的验证产品的功能性和错误处理能力等。测试设计中其中最重要的部分是预见客户对产品的期望及需要,并创建测试用例来验证这些客户的期望。测试设计要从客户的角度上去理解认识产品,并找出产品的不足。
Using Test patterns
如果design pattern一样,test patterns也是用来共通问题,并为测试人员设计测试的时候提供指导方针。
在Testing Object-Oriented Systems: Models, Patterns and Tools, Robert Binder提出了37个test design patterns.
Scope | Pattern Name | Intent |
Method Scope | Design a test suite based on input/output analysis. | |
Combinational Function Test | Design a test suite for behaviors selected by combinational logic. | |
Recursive Function Test | Design a test suite for a recursive method. | |
Polymorphic Message Test | Design a test suite for a client of polymorphic server. | |
Class Scope | Invariant Boundary Test | Identify test vectors based on the class invariant. |
Non-modal Class Test | Design a test suite for a class without sequential constraints. | |
Modal Class Test | Design a test suite for a class with sequential constraints. | |
Quasi-Modal Class Test | Design a test suite for a class with content-determined sequential constraints. | |
Class Scope Integration | Small Pop | Order of code/test at method/class scope. |
Alpha-Omega Cycle | Order of code/test at method/class scope. | |
Flattened Class Scope | Polymorphic Server Test | Design a test suite to check LSP compliance of a polymorphic server hierarchy. |
Modal Hierarchy Test | Design a test suite for a hierarchy of modal classes. | |
Reusable Components | Abstract Class Test | Develop and test an implementation of an interface. |
Generic Class Test | Develop and test an implementation of a parameterized class. | |
New Framework Test | Develop and test a demo application of new framework. | |
Popular Framework Test | Test changes to widely used framework. | |
Subsystem | Class Association Test | Design a test suite based on class associations. |
Round-trip Scenario Test | Design a test suite for aggregate state-based behavior. | |
Controlled Exception Test | Design a test suite to verify exception handling. | |
Mode Machine Test | Design a test suite based on sequentially-constrained stimulus-response scenarios. | |
Integration | Big Bang Integration | Try everything at the same time. |
Bottom up Integration | Integration by dependencies. | |
Top Down Integration | Integration by control hierarchy. | |
Collaboration Integration | Integration by cluster scenarios. | |
Backbone Integration | Hybrid integration of subsystems. | |
Layer Integration | Integration for layered architecture. | |
Client/Server Integration | Integration for client/server architecture. | |
Distributed Services Integration | Integration for distributed architecture. | |
High Frequency Integration | Build and test at frequent, regular intervals. | |
Application Scope | Extended Use Case Test (pdf) | Develop testable use cases, design a test suite to cover application input-output relationships. |
Covered in CRUD | Exercise all basic operations. | |
Allocate Tests by Frequency | Allocate system test effort to maximize operational reliability. | |
Regression Test | Retest All | Rerun all tests. |
Retest Risky Use Cases | Rerun tests of risky code. | |
Retest Profile | Rerun tests by frequency of use. | |
Retest Changed Code | Rerun tests for code that depends on changes. | |
Retest Within Firewall | Rerun tests for code that is impacted by changes. |
附加17种自动测试的test patterns。
Capability | Pattern Name | Intent |
Built-in Test | Perform automatic verification of super/subclass contracts. | |
Test Cases | Test Case/TestSuite Method | Implement a test case or a test suite as a method. |
Catch All Exceptions | Test driver generates and catches IUT's exceptions. | |
Test Case /Test Suite Class | Implement test case or test suite as an object of class TestCase. | |
Test Control | Server Stub | Use a stub implementation of a server object for greater control. |
Server Proxy | Use a proxy implementation of a server object for greater control. | |
API/Class Drivers | TestDriver Super Class | Use an abstract superclass for all test drivers. |
Percolate the Object Under Test | Pass the object under test to driver. | |
Symmetric Driver | Driver hierarchy is symmetric to hierarchy of classes under test. | |
Subclass Driver | Driver is a subclass. | |
Private Access Driver | Driver uses encapsulation avoiding features. | |
Test Control Interface | Driver uses interface extension features. | |
Drone | Driver is a mixin. | |
Built-in Self Test | Driver is implemented as part of an application class. | |
Test Execution Control | Command Line Test Bundle | Code and build a test executable to be run from a command line or console. |
Incremental Testing Framework | Test suites are based on a simple framework that supports incremental development. | |
Fresh Objects | Test environment with registration by and interface to built-in test in all application objects. |
最常用的共享test patterns的方式就是构建模板。
Robert Binder’s Test design pattern 包含10个属性:
1) Name
2) Problem:简要介绍pattern要解决的问题
3) Analysis:介绍问题的区域,并说明为什么采用这个pattern的技术去设计测试更有效
4) Design:说明如何根据pattern设计测试
5) Oracle:说明预期的结果
6) Examples:举例说明使用pattern是如何找bug
7) Pitfalls and Limitations: 说明在某些情况下应该避免使用这个pattern
8) Related Patterns:列出相关联的patterns