引用

Read succesful using of TestLink integrated with other test automation tools by William Seppeler.

I'm a test engineer for Performance Technologies (www.pt.com).  I use Selenium for browser testing and SNMP and Expect for pretty much everything else.  Perl is the test scripting language of choice to tie it all together.  We use TestLink to write our testcases and keep everything organized.  With Perl, I output Test Anything Protocol (TAP).  I then use a simple script to convert TAP to XML, and I import the results directly into TestLink.  The process works great for us. For anyone using Perl with TestLink, it might be useful.

Background

My expertise is mostly automated testing using Perl.  I automate network equipment via SNMP as well as console terminals.  For automating console terminals, I use the Expect library.  Perl provides modules for both SNMP and Expect and has become our number one choice for test scripting language.  Recently, I've been automating a product driven via Web UI.  Selenium has proven to be indispensable for doing web based testing.  Lucky for me, Selenium can be driven via Perl as well.  Perl has been the testing script language that ties it all together.

I use Perl's built in Test Anything Protocol (TAP) to record test results.  It's crude, but simple to use.  TAP has seen some acceptance outside the realm of Perl, but it's main following seems to be within the Perl community.  If you're familiar with Perl, you may have seen TAP in action if modules were installed from CPAN.  During a module installation, the library routines are tested after installation via a series of 'ok' tests.  Although TAP is generally used for unit testing in Perl, I use TAP to test a Device Under Test (DUT) which is defined to be pretty much anything.

A Perl TAP script is like any other Perl script except it outputs a sequence of 'ok' and 'not ok' statements followed by a declaration of total number of tests.  A TAP script is generally denoted with a suffix of '.t'.  One or more TAP scripts are generally found in a 't/' directory.  A TAP script can be run directly as a Perl script, however it's more common to run multiple TAP scripts via a Perl TAP::Harness.

Implementation

In TestLink, I define a testcase I'd like to perform.  I automate a testcase using one or more Perl TAP scripts.  Those test scripts are place in a directory named in reference to the testcase.  I then place all my testcase directories under a single 't/' directory.  This umbrella becomes my testsuite.  I may have common libraries for a testsuite.  I place those in a 'lib/'.  Appropriately, my log files get place in a 'log/' directory.  The 't/', 'lib/', and 'log/' directories are found under a directory named for the testsuite.  As an example:

<testsuite>/
  /lib
  /t
    /<testcase1>
      01-setup.t
      02-test.t
      03-cleanup.t
    /<testcase2>
      01-setup.t
      02-test.t
      03-cleanup.t
  /log
    /t
      /<testcase1>
        01-setup.t
        02-test.t
        03-cleanup.t
      /<testcase2>
        01-setup.t
        02-test.t
        03-cleanup.t

To run my TAP tests, I change into the <testsuite> directory and type:

  prove -r PERL_TEST_HARNESS_DUMP_TAP=log

The 'prove' command is the Perl command line utility to run a TAP::Harness.  By default, the utility looks for tests in the 't/' directory.  The '-r' causes the harness to work recursively.  TAP results generally get sent to STDOUT, but I like to keep a log of all my test scripts.  Therefore I choose to specify a PERL_TEST_HARNESS_DUMP_TAP directory.

After my TAP scripts are run, I then parse the TAP logs in the PERL_TEST_HARNESS_DUMP_TAP directory to generate a results.xml file which I then import into TestLink.  The process is relatively easy.  The only tools I use are Perl, the respective Perl modules from CPAN and this basic directory structure.

As an improvement to this basic test framework, I use the TAP::Formatter::HTML module when parsing my TAP logs to generate an index.html file for each <testcase> directory.  I also make use of TAP's TODO and SKIP syntax to track bugs.  This is all done during the post processing of TAP logs.  The sequence of steps to run a test is:

  1) Execute the TAP scripts (ie: prove -r)
  2) Parse TAP logs to generate HTML and XML.  While parsing, if a <testcase> fails, check if a ticket exists in the bug tracking system.  If so, then mark the <testcase> as a TODO item.  TODO items show as BLOCKED test results in TestLink.

To make the testing process even easier, I have a 'test.pl' Perl script to facilitate the steps above.  Automating tests with Perl then becomes:

  1) Change into the <testsuite> directory
  2) Run 'perl test.pl'
  3) Review test results with browser pointing to log/index.html (optional)
  4) Import test results into TestLink via results.xml file.

 

Summary

Unlike other testing products, the only tools used are Perl along with some common modules from CPAN.  I use a basic test directory structure to mimic my test design in TestLink.  Because the automated testsuite and test logs are all self contained within a single directory, packaging for a single build is a simple as tarring the testsuite directory.

Reference

Download a test.tgz package showing a working example. (Will be added soon.)

The working example would rely on Perl as well as some common library modules from CPAN.  The test framework consists of the 'test.pl' wrapper script (ie: run test then parse logs).  It also contains a lib/TEST.pm script which gets sourced at the start of every TAP script.  The purpose of which is to provide common functionality to all TAP scripts (ie: parsing command line arguments, setting defaults, and so on...).  And a sample TAP script template, which is essentially:

package TEST;

use strict;
use warnings;

use Test::More;

BEGIN {
    unshift(@INC, 'lib');
    use_ok(__PACKAGE__) || BAIL_OUT;    # sources lib/TEST.pm
}

END {
    cmp_ok($?, '==', 0, "exit status");    # verify test completed
    done_testing;
    printf("running time was %d seconds/n", time - $^T);
}

#
# Start Testing
#

1;

 

Discussion

You can discuss the article in our forum: http://www.teamst.org/phpBB2/viewtopic.php?p=8081