Web Service Load Test Strategies

Overview

This document provides information on various ways to perform web service load test. Also, provides information on each type of load test goals.

Test Case Selection

In order to perform a good load test, selecting test case(s) is critical. Test case selection should include a combination of sanity and functional tests of web service request. Each selected test cases must verify all required fields/nodes in the web service response.

Sanity test case tests whether a web service request is returning valid response based on requirements or not.
Functional test case tests whether combination web service request returns desired response or not.

NOTE: In web service load test strategy, it is not necessary to have many different test case scenarios but it is necessary to have test case that covers most required fields/nodes

Strategies

Here are few web service load test strategy reference. These strategies focuses on web service behavior based load test.

Simple: Ramp up number of threads to apply load or run longer for soak test.
Fixed Rate: Simple strategy with TPS (Transactions Per Second) configured. It does not matter how long a web service request takes, next transaction request will automatically start based on configured TPS.
Variable Load: This comprises of following strategies –
1. Variance: Vary the number of thread web service request over load test runtime.
2. Burst: This takes variance to its extreme. Apply large number of thread simultaneous web service request for certain period of time continuously and stop for few milliseconds and apply again.
3. Thread: This linearly changes the number of thread simultaneous web service request from one level to another over the run of load test.
4. Grid: This is advance version of Burst Strategy. Configure number of simultaneous web service request continuously over time without any stop.

Goals

Before designing web service load test, we must ask ourselves what do we want to achieve via web service load test. Here are some goals that web service load test can provide -

Determine web service baseline performance metrics. Choose Simple strategy.
Determine how physical server (where web service is running) recovers from serving web service in between load. Choose Burst or Grid strategy.
Determine how web service handles and recovers when request congestion occurs. Choose Fixed Rate strategy.
Determine at which level certain statistics change or events occur. Choose Thread strategy.
Determine web service stress level. Choose Burst or Grid strategy.

Details From SoapUI Site:

Load Test Concepts: http://www.soapui.org/Load-Testing/concept.html
Simulating Load Strategies using SoapUI Tool: http://www.soapui.org/Load-Testing/simulating-different-types-of-load.html
SoapUI Load Test Execution & Calculations: http://www.soapui.org/Load-Testing/loadtest-execution.html