The future of testing-CSDN博客

By Kingston Duffie

September 1, 2009 —

What if testers were able to somehow work ahead of developers? I don’t mean agile testing, I mean anticipating builds and developing solid test plans well ahead of time. What if instead of developers sending builds to testers, and testers sending bugs to developers, we had testers sending automated tests to developers, and developers shipping products when those tests passed on a candidate build? If we had all this, we would be looking at the future of testing.

You are probably thinking that this is completely unfeasible. How could we possibly get all of the testing automated in time to give to the developers while they are still creating the features? Or you might be thinking that this is crazy, because you can’t trust developers to release the product themselves! But bear with me. Let’s consider the fundamentals of this concept before we start to deal with the issues.

In this new process, things get very efficient. Testers spend their time designing great tests. Since that is all they are doing, they are going to produce more tests with better coverage. Developers are going to be a lot more efficient, too. They have tests to tell them immediately when they break something, rather than dealing with the vagaries of bug reports that come much later.

In most cases, test failures are much easier for the developer to analyze given their understanding of the recent code changes. The most important thing in this process is the absence of any “looping.” We eliminate the expensive process of bug fixes requiring new builds that then need to be re-tested, producing new bug fixes, and so on. The product is ready to release when all of the tests have been completed and when all of those tests pass.

Also, this new process workflow could solve a common challenge for many organizations. QA testers often lack sufficient knowledge or understanding of how the features that they will be testing should work. As a result, the majority of test development occurs after QA finally gets to the see the feature in action from an early build. In theory, they should be able to build a solid test plan—and test cases—from the specification provided by the developers and use cases from product management. However, this rarely happens: Either the spec changes or the testers are not knowledgeable enough to anticipate testing, so they wait.

Given all this, what would it take to make this process feasible?

1. You must be able to automate all tests.
2. You must be able to automate tests without having the product/feature available.
3. You must be able to automate in less time than it takes to develop the product/features.
4. You must be willing to change.

Let’s take these one at a time.

Is it feasible to automate all tests? Perhaps not (although that might be an interesting debate). Nevertheless, let’s suppose that you can automate 95% of your testing. Your team can then handle that last 5% using a smaller ancillary process.

Is it feasible to automate feature- and system-level tests without having access to the product? Theoretically, yes. But we know that in practice one must debug automated tests much like one debugs software. So the absence of a real working version of the product or feature is problematic. However, certain virtual technologies now make it possible to emulate the behavior of a system, so you can specify how a device will respond and therefore debug your test case in advance of having access to the final product or feature.

Can test automation happen ahead of development? The answer is definitely yes. Automated testing tools can dramatically accelerate the process of automation to the point where the first few tests on a new feature can be in place in a matter of hours. Additional tests extending that coverage can be developed in parallel as development proceeds, ensuring that the developer always has a good set of tests to work with.

By moving automated testing ahead of development, organizations stand to benefit by eliminating looping between QA and developers. Today, developers test features with a positive use in mind—that is, they test simply to prove that a feature works as designed. QA teams, on the other hand, conduct negative use tests to see how the feature responds when users use it incorrectly.

In the standard workflow, tests flow from development to QA—first positive use, then negative use—which leads to looping when defects are found. However, if QA testers provided developers with automated negative use tests, developers could then run both types of tests and avoid looping. This would save developers as well as testers considerable time and effort.

To implement this change to the workflow, testers and developers need a technology that enables them to easily transfer, understand and run tests. The resulting asset sharing would not only eliminate looping, as mentioned earlier, but also lead to efficiencies throughout the workflow. For example, QA testers would be able to pass on automated tests to developers; developers would leverage these tests to conduct feature/unit testing; and then developers would pass these semi-automated tests to the automation specialists. Automation specialists could easily leverage these semi-automated test cases (they might not have pass/fail criteria) for regression-validation testing, eliminating the need to decipher written test plans or build automated tests from scratch.

Will organizations change their processes? I understand that these kinds of changes don’t happen quickly. But these changes will come. As with the advent of agile development, making this change requires business drivers and technology enablers, both of which exist in the form of market pressures and automated testing tools. The early adopters will be those who feel the greatest pain and are willing to take some risks to change. Others will follow.

The future of testing is really about stepping outside our comfort zones and helping developers and testers work together in a new and improved way. It’s about being innovative behind the scenes and rethinking familiar processes in order to deliver higher-quality products to market faster.

Kingston Duffie is founder and CTO of Fanfare Group, which provides software quality assurance products and services.