Staying Out of Deep Water: Performance Testing Using HTTPD-Test's Flood

Testing 'News' Style Web Sites

The majority of news-style Web sites -- New York Times, Slashdot, ServerWatch, and even many blog sites -- have a main index page, which everybody accesses, that contains the links to the main stores and typically one or more 'story' pages when a user deems the story interesting enough to read in full.

In general, this results in a fairly steady stream of people hitting the main page and a variable number hitting specific other pages. If a site publishes an RSS/RDF feed it will also see a fair number of accesses directly to story pages without first viewing the home page. Nearly all of these types of site use dynamic elements, and using Flood is also a good way to test the dynamic performance of your site, especially if you can compare that to a raw HTML-based response.

You can simulate the news style requests through Flood by using the following settings:

Testing Shopping Sites

Shopping sites, online product catalog, and other more interactive sites have a different usage profile. Although some people will hit your home page, some will come in directly to another page within your site. Most users will also spend more time browsing around the site -- they'll look at a product page for a number of products, perhaps do some searches and even click through to other related or similar products.

Thus, you should test with a higher number of URLs in the list, larger repeat counts (to simulate a larger number of users), and lower simultaneous access than with news site:

Testing the "Slashdot" Effect

Occasionally, a situation will arise where you must check how your system copes with thousands of users trying to access the site at the same time. Many sites have already experienced the "slashdot" effect, whereby a mention of a particular page on the Slashdot Web site (www.slashdot.org) results in these large, simultaneous, requests.

Typically these requests are for only one page, and we can test for that with Flood by creating hundreds, or thousands, of farmers simultaneously accessing the server for just that one page. To simulate the rapid sequential access by a number of readers over a period of time, set a high repeat count and use the delay system to ramp up the requests to their high point.

Tips for Testing

For any of these tests to work properly, you must keep a few things in mind:

• First and foremost, don't run Flood from the same machine as your HTTP server. If you do this you're only testing a single machine's capability to open network sockets and communicate with itself. Test from another machine on the network.
• Keep in mind the technical limits of the machine you test from. Your machine is capable of only so many threads and network sockets; trying to create too many farmers may result in inefficient and misleading testing.
• Flood is a client-side solution, so nothing is stopping you from executing the tests simultaneously on multiple machines. In fact, we recommend that, as it's the only way to reliably flood test a server to the point of saturation before reaching the client's own limits.
• Flood is reliant on the client host for performance. If the client test machine is busy, Flood will have just as many hurdles as any other application in terms of processing requests. In dedicated Web farm environments I've set a single machine whose sole responsibility is testing. If you are using a machine normally devoted to other tasks, shut them down before starting Flood.

Future articles will examine ways to summarize the report information and how to test more complex sites and servers.