A Simple Interpretation of the 4GWM

A Simple Interpretation of the 4GWM
2006-9-26

The 4th generation white-box-testing methodology (4GWM) has been published in the Internet for some time. Recently, some friends from the Internet asked me what the major differences between the 4GWM and other methodologies are. Some also pointed out that the 3rd generation methodology has explicit boundary with the 2nd generation one, but the 4th generation one is not the case with the 3rd generation one. Herein, we’re about to answer these questions and supplement the intension and extension of the 4GWM. This article may serve as the supplementary reading material for the Introduction to the 4GWM (Theory).
   
1. From the 1GWM to the 4GWM
Major difference among 1GWM ~ 4GWM:

	Test Effect Assessment	Automatic Testing	Continuous Testing	All-in-One Debugging and Testing
1GWM	No	No	No	No
2GWM	Yes	Yes	No	No
3GWM	Yes	Yes	Yes	No
4GWM	Yes	Yes	Yes	Yes

The 1GWM belongs to an unorganized, unlimited testing methodology. A typical of the 1GWM is adopting debugging for testing or simply adding print, assert, etc. statements in the codes under test. The 2GWM has mainly conquered two major defects in the 1GWM:1) the 1GWM offers no test assessment (e.g. coverage); 2) test operations cannot be reused in the 1GWM. In order to reuse test operations, a formalized language must be used to describe the testing process. Such formalized description falls into: 1) using the language used to develop the codes under test; 2) using an abstracter and easier script language.

Popular 2GWM tools include: Rational’s RTRT, Parasoft’s CppTest, and IPL’s Cantat++.

The 3GWM is represented by xUnit series testing tools, such as JUnit, NUnit and DUnit.The most important feature of the 3GWM that differs from the 2GWM is that the 3GWM supports the continuous integration operating mode, which is a great leap in the methodological philosophy.

The 4GWM has inherited the continuous testing requirements from the 3GWM. Such inheritance is inclusive, just as the 3GWM has inherited formalized testing description from the 2GWM. On the basis of the 3GWM, the 4GWM has stressed the reuse of operations by reusing debugging operations in testing. As for whether this upgrade has any important meaning and whether the 4GWM deserves a upgrading, we’ll detail these questions below. Firstly, let’s make clear a concept: compared with the 3GWM, the 4GWM has mainly introduced the philosophy of all-in-one debugging and testing.Testing tools following the 4GWM are represented by VcTester (http://www.ezTester.com).

2. In Assessing the White-Box-Testing Methodology, Test Efficiency Improvement Shall be First, and Then the Test Quality Improvement
To assess a testing tool, we may need to check a lot of factors, such as the software usability, whether it can improve work efficiency, whether it can propel the testing to deeper level, etc. However, as for most enterprises nowadays, the assessing factors are not equal. If an enterprise has never implemented the white-box testing and desires to promote it all of a sudden, it must make sure whether a selected tool is highly efficient enough at first, instead of checking whether it supports more types of coverage (e.g. turn to another tool if the current one does not support MCDC coverage assessment).

In fact, although the white-box testing is very important, its practice is still not so meaningful for most enterprises, especially for projects developed with C language. The bottleneck does not lie in whether the white-box testing can be done well, but in whether the white-box testing can go on! According to our promotion experiences in this field,among projects with failed white-box tests, over 95% is because the testing cannot go on due to low testing fficiency and thus high costs. The major cause of failure is the discontinued implementation of testing. In other words, in current status, the basic problem is whether the white-box testing can be ensured to go on, and the performance of implementation is the secondary question. After the testing is carried out, we can focus on the improvement. The former question is the essence which has the final say; the latter one is for a better work, and is additional blessing for most enterprises in the current status since it is easier to address.

We can verify the above attitude from another aspect. As we all know, JUnit and NUnit works well in Java and C#, and there are many successful cases. However, there are much fewer cases as for CppUnit in C++, and even fewer as for CUnit in C++ (those few successful ones are due to big investment and good organization). It is not hard to tell that testing tools based on the same methodology have different implementation effects in different languages. The major reason is that tools differ greatly in improving test efficiency.Since Java and C# belong to high abstract level, the efficiency is high even when using them to write, debug and maintain cases. On the contrary, as for process programming languages, such as C, it is hard to succeed when it is used describe the test operations.

3. From the 1GWM to the 4GWM, with Enhancing “Reusability”Step-by-Step
From the 1GWM to the 2GWM, the testing can be described in a formalized method, so that testing operations can be recorded in scripts (or some language) to support repeated testing. A testing operation can be used repeatedly in later testing, which greatly improves
the test efficiency.

From the 2GWM to the 3GWM, the continuous integration has realized the reuse of software stability. How? The 3GWM requires an iterative mode of “writing some and testing some” to propel project R&D, so that the codes under test remain relative stability in every round of iteration and newly emerging bugs can be detected and located immediately by analyzing recent modified scores of code lines. In such operating mode,the bug detection and rectification efficiency is very high. In essence, the continuous integration mode has realized the reuse of “software stability”. The direct benefits are embodied in: 1) the bug detection and rectification efficiency is enhanced; 2) the system under test is always in an operation-ready status, so that the functionality to be achieved can be shown in advance, which can effectively lower the R&D risks.

From the 3GWM to the 4GWM, after the introduction of the philosophy of integrated debugging and testing, testing codes are truly treated as a type of products codes equal to the codes under test, and they are added and maintained at the same time. It’s not necessary to design test scripts after completing the entire codes under test. The maintenance is also equivalent. If one code is modified, the associated one shall be modified as well. What’s more, the debugging processes of testing codes and tested codes can also be integrated, since there are many common characteristics and many reusable places. For example, we need to construct a running environment before debugging the codes under test, so is the case in debugging test scripts. In debugging, we often need to modify and inspect variables so that the following single-step trace can follow a particular path, while in testing, we need also to modify variables or post test stubs to cover particular paths; in debugging, we need to manually check whether a variable works, while in testing, we use case scripts to determine whether the testing is passed. If debugging operations are reused in testing, or, to be specific, if debugging and
testing are integrated according to the philosophy of the 4GWM, the efficiency in software R&D will be greatly improved undoubtedly.

The white-box testing for embedded software mainly aims at C language, which is a hard domain for xUnit series. However, due to the reusability of debugging operations,VcTester can well conquer this obstacle. In addition, CSE script language can well emulate behaviors of C language, so as to improve the test case writing and debugging efficiency. The market feedback of VcTester has well evidenced this. The VcTester users reflect that VcTester’s efficiency in developing cases is far higher than other tools. There are two factors propelling the improvement of efficiency: 1) online testing; 2) reusable debugging operations. The contribution of the latter is not only the convertibility from debugging operations to scripts, but also the test design mechanism of “one driver for multiple cases” supported by VcTester inspector, which can greatly save manpower investment in case writing.

4. Other Features of the 4GWM
The 4GWM has explicitly stipulated three key fields including 9 key features, as follows:
A. The 1st key filed: online testing
1) Online testing driven
2) Online script stub
3) Online test case design, running, and assessment & improvement
B. The 2nd key field: grey-box debugging
1) Call-based interface
2) Debugging is testing
3) All-in-one coding, debugging and testing
C. The 3rd key field: continuous testing
1) Test design first (TDF)
2) Continuous guaranteed confidence
3) Test design reconstructing

With the understanding of the philosophy of all-in-one debugging and testing mentioned above, it will be easier to understand the 4GWM. Online testing is a measure to enable the testing mini-cycle and propel the realization of the R&D big-cycle. The measure is a convenient way to improve the work efficiency. Grey-box testing is actually a practice and presentation of the philosophy of “all-in-one debugging and testing”. The continuous testing actually regulates the philosophy of continuous integration conformed by the 3GWM in an express way In addition, the continuous testing in the 4GWM attempts to
conquer difficulties incurred in test driver development (i.e. TDD practices) in XP. After all,the 4GWM featuring online testing, integrated debugging and testing, etc. can greatly address the troublesome mess in writing case before the coding work.

Reference:
1. ezTester, Wayne Chan, Introduction to the 4GWM (Theory)
2. ezTester, Wayne Chan, Introduction to the 4GWM (VcTester Practices)

欢迎转载此文，转载时请注明文章来源：文斯测试技术研究中心 http://blog.csdn.net/vincetest