magento siege

Laying Siege to Magento: Performance testing your Magento webstores

December 6, 2010

As I alluded to in my excuses for not posting in ages blog post the other day I am working on a newMagento hosting review for a company called MageMojo. In doing so I’m trying out the performance testing tool called Siege. Siege was mentioned several times at the Magento Developers Paradise, so I have been wanting to try it ever since.

This is a little article that summarizes the steps I took to do the testing, in case you want to test your own Magento stores.

Update: I have since created a tool that will allow you to run testing like this from the comfort of your own web browser. Check out the automated Magento performance testing atMageSpeedTest.com.

Get a Server

Being all the way over in New Zealand pretty much means performance testing Magento from here is out of the question. I fired up an Amazon EC2 small instance for the purposes of testing for a few hours, 20c – I’ve said it once, I’ll say it again, commodity computing is the future . I won’t bore you with a tutorial on starting an EC2 server, there are enough perfectly good guides already online.

You could optionally use a server you already have, or run this from your own computer at the office or home. It would arguably be a fairer test of latency, not running the tests in a production data centre!

Definitely do not run it from the server that is running Magento, that would be a poor test; a) because there would be no latency and b) because the server would be using resources to run the tests, and serve the webstore, that’s not very fair on it!

Install Siege

wget http://www.joedog.org/pub/siege/siege-latest.tar.gz   tar xvzf siege-latest.tar.gz   cd siege-2.71/   # if you're on a fresh ubuntu, get this package first sudo apt-get install build-essential   ./configure   make && sudo make install

Now you can begin a siege on your store, but first, an important warning…

If you use Siege on your production store, be careful it does not backfire.

Too many requests on your production server could easily make it unusable for other visitors, so be careful. Start small, with low numbers of concurrent users for short durations and keep an eye on your servers memory usage. You can halt an in-progress siege with ctrl+c at any time.

Oh and I’m sure the people who read my blog don’t need to be told this but; if you use Siege maliciously to _performance test_ other people’s websites (from multiple Amazon EC2 instances) that’s not performance testing anymore – that’s a denial of service attack, I strongly advise against it!

Run a Basic Siege Test

Now you can run your first Siege test. In this test we just hit one URL with some number of concurrent users (10) for some amount of time (1 minute):

siege -c10 -t1M http://www.google.com

Run a More Complex Test against Magento

First grab a bunch of categories and products from the sitemap.xml (you have a sitemap, right?) and put them into a urls.txt file.

#categories curl http://yourmagentostore.com/sitemap.xml | sed 's/\<url\>/\<url\>\n/g' | grep 0.5 | sed 's/.*loc>\(.*\)<\/loc.*/\1/g' > urls.txt #products curl http://yourmagentostore.com/sitemap.xml | sed 's/\<url\>/\<url\>\n/g' | grep 1.0 | sed 's/.*loc>\(.*\)<\/loc.*/\1/g' >>urls.txt

And then run the test like this for 30 seconds with 25 concurrent users, adjust to suit:

siege -i -c25 -t30s -f urls.txt

Note: I bet a 100 million dollars there is a more elegant way to grab the urls from a sitemap. Someone post one in the comments so I can buy an island.

Looking at Results

Siege results look something like this:

Transactions: 149 hits Availability: 100.00 % Elapsed time: 13.87 secs Data transferred: 1.26 MB Response time: 0.37 secs Transaction rate: 10.74 trans/sec Throughput: 0.09 MB/sec Concurrency: 4.02 Successful transactions: 149 Failed transactions: 0 Longest transaction: 0.42 Shortest transaction: 0.36

The most import numbers to look for are Transaction rate and Response time they will give you an indication of how quickly your server is responding and how many connections it can handle per second.

I hope this little demo has been helpful. I’ll be following up with the full test results for MageMojolater this week, I also compared the Nexcess results – very interesting.

I’d be very interested to hear from anyone with tips and ideas for performance testing, likewise if you try this and have any trouble feel free to ask for help.

Edit: Pefromance…doh….
Edit: Updated Siege URL and version, thanks to Eric@MageMojo for letting me know.

You might also be interested in:

1. MageSpeedTest.com with Siege – Performance Testing Magento Made Easy
2. Pre-release: SMTP Pro / Magento Gmail & Google Apps Email Beta testing (win a prize)
3. Introducing MageSpeedTest.com V2 – Performance test Magento from 4 different countries
4. New Version of Magento Cloud Backup Available: bug fixes, stability and self testing.
5. Java App Engine integration testing with Spring MVC and OpenPersistenceManagerInViewFilter