Linux camp slashes out at ‘NT beats Linux’ survey

Microsoft paid for the 'maimed' survey, now it looks like Mindcraft is going to pay for it too...

  • alert
  • submit to reddit

The essential guide to IT transformation

A Microsoft-sponsored test has 'proved' that the combination of Windows NT and Microsoft Internet Information Server outperform Linux and Apache as a Web server, and Linux and Samba as an SMB file server. Unsurprisingly the survey has come under withering (and, it appears, justified) fire from the Linux camp. The test was carried out by Mindcraft, Inc., whose previous greatest hits include providing evidence that supported Microsoft's claims that it was all Apple's fault that Quicktime didn't work with Internet Explorer. The outfit concludes that NT 4.0 is 2.5 times faster than Linux as a file server and 3.7 times faster as a Web server. Presumably this means that an awful lot of customers out there (including The Register, which scores 100 on the Dumbo scale by running Linux and Apache) are just plain stupid. NT isn't exactly the world's Web server of choice, whereas Apache is. Mindcraft's results somewhat cheekily suggest that NT is more scalable than Linux, and show NT's NetBench throughput starting to climb above Linux at 32 users. We could note here that even on Microsoft's figures this indicates that Linux is a more appropriate fileserver for most small businesses than NT, but the Linux people are already pointing out that Mindcraft's Linux installation was maimed. We'd be the last to suggest this was deliberate of course. If it wasn't the case that the installation was effectively crippled, then the results are still difficult to credit. NT is repeatedly criticised for scalability limitations, and Microsoft itself has repeatedly promised scalability improvements for the product. Steve Ballmer himself did so again, just last week, at WinHEC. Here's another little bit of cheek from Mindcraft: "The Linux 2.2.x kernel is not well supported and is still changing rapidly… We started the tests using Red Hat Linux 5.2 but had to upgrade it to the Linux 2.2.2 kernel because its Linux 2.0.36 kernel does not support hardware RAID controllers and SMP at the same time. In addition, there are comments in the Red Hat Linux 5.2 source code noting that the SMP code is effectively Beta-level code and should not be used at the same time as the RAID driver. For this reason, we upgraded to the Linux 2.2.2 kernel, which has full support for both hardware RAID controllers and SMP to be used simultaneously. As of the date this report was written, Red Hat did not ship or support a product based on the Linux 2.2.x kernel." There's certainly some justification for this, but look who's talking. For inexperienced users it's probably reasonable to say that the installation of a large-scale Linux server system supporting both RAID and SMP is non-trivial, and this isn't helped by the fact that Linux is very much a moving target. But an inexperienced user who tries to do this, rather than hiring experienced Linux support that knows what it's doing, is a maniac. A user without serious NT experience trying to do so is the same. In support of its case, Mindcraft lists six Linux kernel updates so far this year. We know from the Halloween papers that Microsoft really sees this development momentum as a massive advantage for Linux, so flipping it on its head in this way is cynicism of heroic proportions. There is of course absolutely no danger that NT users will be threatened by anything like this rate of change - mind you, those NT service packs just seem to keep coming. The main challenge to the Mindcraft report however comes at the technical level. According to the Linux Weekly News critique: "no adjustments were made to the 'bdflush' and 'file system cache size' parameters supported by the 2.2 kernel," possibly accounting for a factor of two performance improvement in Samba, which was used for the fileserver test, when these are set correctly. The survey also used Windows clients rather than NT - Windows clients will tend to drive the results in favour of NT, while NT clients go better with Samba. LWN continues: "Their Samba configuration sets the widelinks parameter to 'no'. This setting increases the system call overhead for file name lookups considerably. The penalty is especially severe on SMP systems. "Their Apache configuration disables KeepAlive, an important real-world optimisation. (It has been pointed out that the tests do not use KeepAlive in any case). The Apache configuration is also not suited to large loads. It initially starts 10 servers, and MinSpareServers is set to 1. In particular, quick response to sudden, heavy loads will be reduced by this configuration. It is not an 'enterprise' configuration." Eric Green at Slashdot meanwhile points out that the system was probably running out of file handles, and that increasing them would improve performance, while there's also a possibility that running both systems on the same disk (which was the case with the test) would give whichever system was on the outer part of the disk a 1.5-2 times transfer rate advantage. Mindcraft's claims that it searched for support in newsgroups have also been questioned. According to Samba project member Jeremy Allison: "I just did searches on DejaNews in comp.protocols.smb, the samba mailing lists and also in the samba-bugs Jitterbug database (isn't it nice that we keep records of all email we get :-) looking for a request for help from Mindcraft with respect to their benchmark tests. Or even *any* email from Mindcraft. The only Mindcraft references I found were discussions of their previous benchmark trashing Syntax running on Solaris when benchmarked against Windows NT." Register reader Luca Lizzeri however suggests Mindcraft may have sought support under the pseudonym will@whistlingfish.net, but now tells us his eariler suggestion that peter@flyingsaucer.org is a possible ringer was a joke. Great domain name anyway, Peter... ®

Boost IT visibility and business value

More from The Register

next story
6 Obvious Reasons Why Facebook Will Ban This Article (Thank God)
Clampdown on clickbait ... and El Reg is OK with this
No, thank you. I will not code for the Caliphate
Some assignments, even the Bongster decline must
Barnes & Noble: Swallow a Samsung Nook tablet, please ... pretty please
Novelslab finally on sale with ($199 - $20) price tag
Banking apps: Handy, can grab all your money... and RIDDLED with coding flaws
Yep, that one place you'd hoped you wouldn't find 'em
Video of US journalist 'beheading' pulled from social media
Yanked footage featured British-accented attacker and US journo James Foley
Primetime precrime? Minority Report TV series 'being developed'
I have to know. I have to find out what happened to my life
Netflix swallows yet another bitter pill, inks peering deal with TWC
Net neutrality crusader once again pays up for priority access
Judge nixes HP deal for director amnesty after $8.8bn Autonomy snafu
Lawyers will have to earn their keep the hard way, says court
prev story


Best practices for enterprise data
Discussing how technology providers have innovated in order to solve new challenges, creating a new framework for enterprise data.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Advanced data protection for your virtualized environments
Find a natural fit for optimizing protection for the often resource-constrained data protection process found in virtual environments.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?