WebServer Compare: The Yin and Yang of Benchmarks

By ServerWatch Staff (Send Email)
Posted Jul 5, 2000


"Benchmarking software, including Web servers, offers a tantalizing paradox: Measurements are both concrete and tangible, yet at the same time vague and ambiguous. Benchmarks hand us hard numbers, but what do these numbers mean? Often very little, in a broad sense, or sometimes something very specific in a narrow sense."

>The upshot of all this is that Web server benchmarks must be considered in a very limited manner. A benchmark for a server running on a particular operating system, on a particular hardware configuration, focuses only on that specific setup. The further the setup deviates from the tested habitat, the less meaningful the benchmark becomes.

"Web servers are not unlike zoo animals, quite reliant on their habitant. It's simply unrealistic to consider the performance of a specific Web server without regard to its operating environment -- both its software environment, or operating system, and its hardware environment, or the physical components that make up the machine. All of these factors can dramatically influence the final numbers -- how quickly and nimbly a Web server can respond and deliver to network requests."

"What's more, benchmarks can vary widely in how they measure a server. A server's efficiency may also vary depending on the task (e.g., whether its delivering static content to many simultaneous visitors or generating dynamic content)."

"The upshot of all this is that Web server benchmarks must be considered in a very limited manner. A benchmark for a server running on a particular operating system, on a particular hardware configuration, focuses only on that specific setup. The further the setup deviates from the tested habitat, the less meaningful the benchmark becomes."

Page 1 of 1


Comment and Contribute

Your name/nickname

Your email

(Maximum characters: 1200). You have characters left.