基于会话的测试管理

Session-Based Test Management

A method for measuring and managing exploratory testing

 

Description

Exploratory testing is unscripted,unrehearsed testing. Its effectiveness depends on several intangibles: theskill of the tester, their intuition, their experience, and their ability tofollow hunches. But it's these intangibles that often confound test managerswhen it comes to being accountable for the results. For example, at the end ofthe day, when the manager asks for status from an exploratory tester, they mayget an answer like "Oh, y'know... I tested some functions here and there,just looking around." And even though the tester may have filed severalbugs, the manager may have no idea what they did to find them. Even if themanager was skilled to ask the right questions about what the tester did, thetester may have forgotten the details or may not be able to describe theirthinking out loud, on-the-fly.

We had thisproblem when doing exploratory testing for a client. We wanted to beaccountable for our work. We wanted to give status reports that reflected whatwe actually did. We wanted to show that we could be creative, skilledexplorers, yet produce a detailed map of our travels.

How it's done

We inventedSession-Based Test Management as a way to make those intangibles more tangible.It can be thought of as structured exploratory testing, which may seem like acontradiction-in-terms, but "structure" does not mean the testing ispre-scripted. It means we have a set of expectations for what kind of work will be doneand how it will be reported. As in a recording studio, this work is donein "sessions." Sessions range from 45 minutes to several hours, butno matter the length, it is time spent testing against a charter for thesession. The nuts-and-bolts of sessions are described in further detail in an article Jonathan Bachwrote for STQE magazine.

At the end of asession, the tester hands in a session report, tagged with importantinformation about what they did. Here is a sample.

Session metrics

The sessionmetrics are the primary means to express the status of the exploratory testprocess. They contain the following elements:

o  Number of sessions completed

o  Number of problems found

o  Function areas covered

o  Percentage of session time spent settingup for testing

o  Percentage of session time spent testing

o  Percentage of session time spent investigating problems

This is what the metrics looklike.

To create themetrics, information in session files is scanned by a tool we wrote in Perl.

Debriefings

At the end of eachsession, the tester and manager get together to talk about it. We've discoveredthat the value of SBTM relies on the ability of the test manager to talk withthe tester about the work that was done, so to help the tester and manager makethe most out of that meeting (which takes about 15-20 minutes), we've compileda checklist of questions.

The Scan Tool

Our tool"scans" session reports by looking at the tagged headings withinthem. Scans are cumulative, which means information from all of the tags in allof the sessions are collected every time a scan is run.

The prototype isavailable here in aself-extracting executable, but you'll need to install Perl first, which isavailable free athttp://www.activestate.com.

Manager's Guide

We've discoveredthat this methodology relies on the skill of the test manager, so we're workingon a Manager's Guide which will discuss session protocols, the benefits and theproblems we encountered when using SBTM. When that guide is ready, we'll postit here.

Presentations

Jonathan Bachfirst presented Session-Based Test Management at STAR West 2000, in a talktitled"How to Measure Ad HocTesting"

 

 

 

转载：http://www.satisfice.com/sbtm/