最近两年在做Linux虚拟化开源项目,由于项目代码庞大且开发人员分散在世界各个角落,我们对其测试的方法和实践中,对单元测试等做得并不多(KVM的kvm-unit-tests可以算一个吧),对单元测试代码覆盖率等也没有做过多的要求。不过以前做电子商务测试时,那时技术部门曾主推单元测试(主要是JUnit测试Java代码),且将单元测试的代码覆盖率有较为严格的要求,我也做过一小段时间的研究。偶然看到淘宝公直的一篇文章(Linux下c/c++项目代码覆盖率的产生方法),就用他给的实例代码使用了一下gcov和lcov查看了一下C/C++代码覆盖率的统计。
对于Linux下的C/C++代码,可以在gcc/g++工具编译代码时加上“–coverage”选项来生成可用于覆盖率分析的二进制文件,然后用gcov命令来生成覆盖率统计情况,也可用lcov工具来生成html版本的可读性很好的代码覆盖率报告。
下面列举一下各种编程语言的代码覆盖率工具吧。
c/c++: gcc+gcov+lcov;(单元测试:CUnit、CPPUnit、Google GTest等)
Java : Maven cobertura 插件,Clover,EMMA,Jtest;
Python: PyUnit + coverage.py;
PHP: phpunit + –coverage-html + Xdebug ;
Perl: Test::Class 和 Devel::Cover;
Shell: shUnit2 + shcov;
更多的工具资料请参考如下:
代码覆盖率工具: http://en.wikipedia.org/wiki/Code_coverage
单元测试框架:http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks
gcc manual: man gcc (搜索–coverage)
http://ltp.sourceforge.net/coverage/lcov.php
shunit2.googlecode.com
http://sdet.org/?p=212
http://toast.taobao.org/
linux下稳定性测试工具(拷机工具)
stresslinux
super pi
prime
mprime
nbench
cpuburn
gamut
mersenne prime
stress
cpu burn-in
memtester
memtest86
memtest86++
System Stability Tester
BONNIE++/HPL_TEST(不是专门的压力测试软件)
http://www.ocforums.com/showthread.php?t=486495
六款优秀的 Linux 基准测试工具
基准测试是指运行计算机程序去评估硬件和软件性能的行为。硬件基本测试包括评估处理器,内存,显卡,硬盘,网络等不同组件的性能。基准测试有两类: 复合和应用。复合基准对一个硬件执行压力测试,如连续写入和读取数据。应用基准则是衡量真实世界应用程序如数据库和服务器的性能。基准测试软件可以让系统 测试者和用户客观独立的评估硬件性能。Linux下有许多优秀的开源Linux基准测试工具,如Phoronix Test Suite,IOzone,netperf等。
phoronix.com 是业内一个知名的网站,其经常发布硬件性能测评以及 Linux 系统相关的性能测评, Phoronix Test Suite 为该网站旗下的 linux 平台测试套件 , Phoronix 测试套件遵循GNU GPLv3协议。Phoronix Test Suite 默认是通过命令行来的进行测试的,但也可以调用GUI,Phoronix Test Suite 还提供了上传测试结果的服务,也就说你可以把你的测试结果上传在网上,从而可以和别的 Linux 用户测出来的结果进行对比。
iozone 是一款Linux文件系统性能测试工具。它可以测Reiser4, ext3, ext4
Netperf是一种网络性能的测量工具,主要针对基于TCP 或UDP的传输。Netperf根据应用的不同,可以进行不同模式的网络性能测试,即批量数据传输(bulk data transfer)模式和请求/应答(request/reponse)模式。Netperf测试结果所反映的是两个系统之间发送和接受数据的速度和效 率。
Netperf工具是基于C/S模式的。server端是netserver,用来侦听来自client端的连接,client 端是netperf,用来向server发起网络测试。在client与server之间,首先建立一个控制连接,传递有关测试配置的信息,以及测试的结 果;在控制连接建立并传递了测试配置信息以后,client与server之间会再建立一个测试连接,用来来回传递着特殊的流量模式,以测试网络的性能。
LLCbench (底层表征基准测试 Low-Level Characterization Benchmarks) 是一个基准测试工具,集成了 MPBench, CacheBench, 和 BLASBench 测试方法。
HardInfo是一个Linux系统信息查看软件。它可以显示有关的硬件,软件,并进行简单的性能基准测试。
The purpose of this Linux Test Tools Table is to provide the open-source community with a comprehensive list of tools commonly used for testing the various components of Linux.
My hope is that the community will embrace and contribute to this list making it a valuable addition to the Linux Test Project.
Please feel free to send additions, updates or suggestions to Jeff Martin . Last update:07/12/06
Cluster
HINT | allows fair comparisons over extreme variations in computer architecture, absolute performance, storage capacity, and precision. | It's listed as a Past Projectwith a link to http://hint.byu.edu but I have not been able to find where it is being maintained. If you know, please drop me a note. |
Code Coverage Analysis
gcov | Code analysis tool for profiling code and determining: 1) how often each line of code executes, 2) what lines of code are actually executed, 3.) how much computing time each section of codeuses | |
lcov | LCOV is an extension of GCOV, a GNU tool which provides information about what parts of a program are actually executed (i.e. "covered") while running a particular test case. The extension provides HTML output and support for large projects. |
Database
DOTS | Database Opensource Test Suite | |
dbgrinder | perl script to inflict stress on a mysql server | |
OSDL Database Testsuite | OSDL Database Testsuite |
Debug
Dynamic Probes | Dynamic Probes is a generic and pervasive debugging facility. | |
Kernel Debug (KDB) | KDB is an interactive debugger built into the Linux kernel. It allows the user to examine kernel memory, disassembled code and registers. | |
Linux Kernel Crash Dump | LKCD project is designed to help detect, save and examine system crashes and crash info. | |
Linux Trace Toolkit (LTT) | The Linux Trace Toolkit is a fully-featured tracing system for the Linux kernel. |
Defect Tracking
Bugzilla | allows individuals or groups of developers to keep track of outstanding bugs in their product effectively |
Desktop/GUI Libraries
Android | open source testing tool for GUI programs | |
ldtp | GNU/Linux Desktop Testing Project |
Event Logging
included tests | Various tests are included in the tarball |
Filesystems
Bonnie | Bonnie++ is test suite, which performs several hard drive/ filesystem tests. | |
dbench | Filesystem benchmark that generates good filesystem load | |
fs_inode | Part of the LTP: This test creates several subdirectories and files off of two parent directories and removes directories and files as part of the test. | |
fs_maim | Part of the LTP: a set of scripts to test and stress filesystem and storage management utilities | |
IOZone | Filesystem benchmark tool (read, write, re-read, re-write, read backwards, read strided, fread, fwrite, random read, pread, aio_read, aio_write) | |
lftest | Part of the LTP:lftest is a tool/test designed to create large files and lseek from the beginning of the file to the end of the file after each block write. This test verifies large file support and can be used to generate large files for other filesystem tests. | Files up to 2Tb have been created using this tool. This test is VERY picky about glibc version. |
LTP | The Linux Test Project is a collection of tools for testing the Linux kernel and related features. | |
PostMark | Filesystem benchmark that simulates load generated by enterprise applications such as email, news and web-based commerce. | |
stress | puts the system under a specified amount of load | |
mongo | set of the programs to test linux filesystems for performance and functionality | |
fsx | File system exerciser from Apple. | The test is most effective if you let it run for a minute or two, so that it overlaps the periodic sync that most Unix systems do. |
xdd | Storage I/O Performance Characterization tool that runs on most UNIX-like systems and Windows. | Has been around since 1992 and is in use at various government labs. |
Harnesses
Cerberus | The Cerberus Test Control System(CTCS) is a free (freedom) test suite for use by developers and others to test hardware. It generates good filesystem stress in the process. | |
STAF | The Software Testing Automation Framework (STAF) is an open source framework designed to improvethe level of reuse and automation in test cases and test environments. |
I/O & Storage
tiobench | Portable, robust, fully-threaded I/O benchmark program | |
xdd | Storage I/O Performance Characterization tool that runs on most UNIX-like systems and Windows. | Has been around since 1992 and is in use at various government labs. |
Kernel System Calls
crashme | a tool for testing the robustness of an operating environment using a technique of "Random Input" response analysis | |
LTP | The Linux Test Project is a collection of tools for testing the Linux kernel and related features. |
Network
Connectathon NFS Testsuite | This testsuite tests the NFS Protocol | |
ISIC | ISIC is a suite of utilities to exercise the stability of an IP Stack and its component stacks | |
LTP | The Linux Test Project has a collection of tools for testing the network components of the Linux kernel. | |
netperf | Netperf is a benchmark that can be used to measure the performance of many different types of networking. | |
NetPIPE | Variable time bench mark, ie, it measures network performance using variable sized communiation transfers | |
TAHI | Providesinteroperability and conformance tests for IPv6 | |
VolanoMark | A java chatroom benchmark/stress | |
UNH IPv6 Tests | there are several IPv6 tests on this site | |
Iperf | for measuring TCP and UDP bandwidth performance |
Network Security
Kerberos Test suite | These tests are for testing Kerberos clients (kinit,klist and kdestroy) and Kerberized Applications, ftp and telnet. |
Other
cpuburn | This program was designed by Robert Redelmeier to heavily loadCPU chips. |
Performance
contest | test system responsiveness by running kernel compilation under anumber of different load conditions | |
glibench/clibench | benchmarking tool to check your computer CPU and hard disk performance | |
lmbench | Suite of simple, portable benchmarks | |
AIM Benchmark | Performance benchmark | |
unixbench | Performance benchmark based on the early BYTE UNIX Benchmarks | "retired" since about 1997, but still used by some testers |
Scalability
dbench | Used for dcache scalability testing | |
Chat | Used for file_struct scalability testing | |
httperf | Used for dcache scalability testing |
Scheduler
LTP | The Linux Test Project is a collection of tools for testing the Linux kernel and related features. | sched_stress and process_stress |
VolanoMark | A java chatroom benchmark/stress | VolanoMark has been used to stress the scheduler. |
SCSI Hardening
Bonnie | Bonnie is test suite, which performs several hard drive and filesystem tests. | |
LTP | The Linux Test Project is a collection of toolsfor testing the Linux kernel and related features. | disktest |
dt | dt (Data Test) is a generic data test program used to verify proper operation of peripherals, file systems, device drivers, or any data stream supported by the operating system |
Security
Nessus | remote security scanner |
Standards
LSB | Test suites used for LSB compliance testing |
Stream Controlled Transmission Protocol
LTP | The Linux Test Project is a collection of tools for testing the Linux kernel and related features. |
System Management
sblim | The "SBLIM Reference Implementation (SRI)" is a component of the SBLIM project. Its purposes are (among others): (1) easily set up, run and test systems management scenarios based on CIM/CIMOM technology (2) test CIM Providers (on local and/or remote Linux machines) |
Threads
LTP | The Linux Test Project is a collection of tools for testing the Linux kernel and related features. | |
VSTHlite | Tests for compliance with IEEE POSIX 1003.1c extensions (pthreads). |
USB
usbstress | Sent to us by the folks at Linux-usb.org |
Version Control
cvs | the dominant open-source network-transparent version control system | |
BitKeeper | BK/Pro is a scalable configuration management system, supporting globally distributed development, disconnected operation, compressed repositories, change sets, and repositories as branches. | Read the licensing info |
Subversion |
VMM
vmregress | regrssion, testing and benchmark tool | |
LTP | The Linux Test Project is a collection of tools for testing the Linux kernel and related features. | |
memtest86 | A thorough real-mode memory tester | |
stress | puts the system under a specified amount of load | |
memtest86+ | fork / enhanced version of the memtest86 | |
memtester | Utility to test for faulty memory subsystem |
Web Server
Hammerhead | Hammerhead is a web server stress tool that can simulate multiple connections and users. | |
httperf | httperf is a popular web server benchmark tool for measuring web server performance | |
siege | Siege is an http regression testing and benchmarking utility. | |
PagePoker | for loadtesting and benchmarking web servers |