Performance Testing vs. Load Testing vs. Stress Testing

Performance Testing vs. Load Testing vs. Stress Testing

性能测试 vs. 负载测试 vs. 压力测试

If you’re a developer, QA engineer or DevOps, you know you need to conduct different types of performance tests, to make sure each code change or feature added (a) doesn’t break the system, and (b) works.

假设您是一名开发者、QA工程师或者开发运维，那您就需要进行不同类型的性能测试，以确保每一行代码的更改、新功能的添加都不会让你的系统停止运行。

But which types of performance tests should you conduct, what’s the difference between load tests and other types of tests and which test is suitable for which situation? In this blog post we’ll cover the answers to these questions, and more.

What is Performance Testing?

Performance testing is the general name for tests that check how the system behaves and performs. Performance testing examines responsiveness, stability, scalability, reliability, speed and resource usage of your software and infrastructure. Different types of performance tests provide you with different data, as we will further detail.

Before performance testing, it’s important to determine your system’s business goals, so you can tell if your system behaves satisfactorily or not according to your customers’ needs.

After running performance tests, you can analyze different KPIs, such as the number of virtual users, hits per second, errors per second, response time, latency and bytes per second (throughput), as well as the correlations between them. Through the reports you can identify bottlenecks, bugs and errors, an decide what needs to be done.

When should you use Performance Testing?

When you want to check your website performance and app performance, as well as servers, databases, networks, etc. If you work with the waterfall methodology, then at least each time you release a version. If you’re shifting left and going agile, you should test continuously.

This is an example of a performance test report on CA BlazeMeter. This is a good test, as the growing number of users doesn’t affect the response time, the error rate continues to be low and the hits per second rises with the number of users but remains stable.

What is Load Testing?

Load testing is testing that checks how systems function under a heavy number of concurrent virtual users performing transactions over a certain period of time. Or in other words, how systems handle heavy load volumes. There are a few types of open-source load testing tools, with JMeter being the most popular one.

When should you use Load Testing?

When you want to determine how many users your system can handle. You can determine different user scenarios that let you focus on different parts of your system, like the checkout webpage on your website or app for web load testing. You can also determine how the load behaves, like the geo-location users come from or how the load builds and sustains in the system. Basically, load testing is something you should do all the time, to ensure your system is always on point. That’s why it should be integrated into your Continuous Integration cycles, with tools like Jenkins and Taurus.

This is what a load test would look like in JMeter. This test analyzes adding 100 users every 30 seconds until reaching 1,000 users. Each step takes 30 seconds to complete and JMeter waits 30 seconds before starting the next step. After reaching 1,000 threads all of them will continue running and hitting the server together for 5 minutes.

What is Stress Testing?

Stress testing is testing that checks the upper limits of your system by testing it under extreme loads. The testing examines how the system behaves under intense loads, and how it recovers when going back to normal usage, i.e are the KPIs like throughput and response time the same as before? In addition to load testing KPIs, stress testing also examines memory leaks, slowness, security issues and data corruption.

Stress testing can be conducted through load testing tools, by defining a test case with a very high number of concurrent virtual users. If your stress test includes a sudden ramp-up in the number of virtual users, it is called a Spike Test. If you stress test for a long period of time to check the system’s sustainability over time with a slow ramp-up, it’s called a Soak Test.

When Should You Use Stress Testing?

Website stress tests and app stress tests are important before major events, like Black Friday, ticket selling for a popular concert with high demand or the elections. But we recommend you stress test every once in a while so you know your system’s endurance capabilities. This ensures you’re always prepared for unexpected traffic spikes, and gives you more time and resources to fix your bottlenecks.

This is an example of what a Spike Test would look like on JMeter. This test analyzes adding 7,000 users at once and then adding 500 users every 30 seconds until reaching 10,000 users. After reaching 10,000 threads all of them will continue running and hitting the server together for 5 minutes.

Using different types of performance tests ensures you are always aware of the issues you have and that you can plan how to deal with them. So don’t give up on any types of tests.

Learn more on how to use JMeter from our free Webinar JMeter Load Testing at Scale and from our blog. JMeter can be enhanced through CA BlazeMeter, with larger scalability and many geo-locations, better reporting and sharing capabilities, multiple integrations and recording options. Request a demo to learn more.