This article is more than 1 year old

Benchmark battles – now Linux beats NT

A 'real life scenario' test by c't mag does a good job of putting things in context

In the wake of the latest in the round of NT versus Linux face-offs German magazine c't has published the results of its own tests of the two operating systems. The c't tests, conducted by Jurgen Schmidt, were intended to assess the two rivals in 'real life' situations of the sort Linux is supposed to be good at. Linux does a lot better than it has in the 'clash of armour' benchmarks we've seen so far, and Schmidt makes a number of eminently valid and sensible observations concerning the real operation of Web servers. (Full c't report) The tests pitted NT 4.0 and IIS against SuSE Linux 6.1 and Apache on a quad 450MHz Xeon Siemens server with two gigs RAM, twin EtherPro 100 boards and a RAID system. One of the first 'real life' differences between the c't and Mindcraft tests was the use of RAID-5 rather than RAID-0. Mindcraft's use of the latter was aimed at performance, while c't went for a more realistic performance/stability compromise. Schmidt also comments on the nature of the Mindcraft test: "Unlike for the Mindcraft test, which required the server to produce its pages through four 100-MBit interfaces we decided on a more realistic scenario. How many web servers actually serve four of these network interfaces? The majority of web servers make do with a 10-MBit interface and even in intranet one 100-MBit board should be sufficient. This was the configuration we chose for our tests. To get an impression of maximum load behaviour anyway, we made the server prove it can handle two Fast Ethernet connections." For serving a static HTML page of 4-8k, the two came out roughly even at 4k, with Linux slightly ahead at 8k. Schmidt notes that both operating systems didn't benefit to any great extent from use of multiple CPUs, although the Linux installation was running kernel 2.2.9, which is better at SMP than 2.2.5. Linux did however performs substantially better in random requests of 1,000,000 4k files. With 512 requesters NT managed to answer 30 per second, and Linux 274. In another test using a CGI Perl script, Linux delivered twice as many pages as NT on a single CPU, and 2.5 times as many with four CPUs. This isn't entirely surprising, as IIS' support for Perl isn't great. NT did however shine when using multiple network boards, and Schmidt comments: "Linux's comparatively bad results when tested with two network boards show that Mindcraft's results are quite realistic. NT and IIS are clearly superior to their free competitors if you stick to their rules." In summary, he feels that "additional CPUs for plain web server operation with static HTML pages are a waste. Even with two Fast Ethernet lines there's only a moderate less than twenty percent increase." The server wasn't needing to work to its full capacity, and the tests were simulating conditions tougher than you'd expect in most real life scenarios. "In SMP mode, Linux still exhibited clear weaknesses. Kernel developers, too, admit freely that scalability problems still exist in SMP mode if the major part of the load comes through in kernel mode. However, if user mode tasks are involved as well, as is the case with CGI scripts, Linux can benefit from additional processors, too. These SMP problems are currently the target of massive developing efforts." In the most relevant, practical areas, Linux and Apache "are already ahead by at least a nose," while if the pages don't come directly from main system memory, they're more clearly ahead. c't was also impressed by the level of support it got from the Linux community. Microsoft was slow to respond to requests for information, while "Emails to the respective [Linux] mailing lists even resulted in special kernel patches which significantly increased performance. We have, on the other hand, never heard of an NT support contract supplying NT kernels specially designed for customer problems." A very sensible report, and well worth reading in detail. ®

More about

TIP US OFF

Send us news


Other stories you might like