This article is more than 1 year old

Benchmarks demolish Apple speed boasts

New kit bested by ancient PIII

As of this morning, thirty one entries have been submitted to the SPEC performance list for the year so far. But if you're wondering why Apple hasn't yet dispatched its latest "workstation class" G4 hardware for examination by the council, in what is the industry's most respected set of benchmark tests, C't has the answer.

The German tech bible has put the latest dual G4s through the SPEC CPU2000 processor benchmark, and the results make dismal reading for hardcore Apple loyalists. C't found that the RISC-based machines running OS X fall severely short of expectations, being bested in the floating point tests by an eighteen month old Pentium III-based machine

You might still be able to buy one of these, on eBay.

In the integer tests, which give a better indication of performance for the kind of general-purpose desktop computing most of use every day, the G4 held its own against the ancient Pentium running at 1Ghz.

But when C't ran the floating point benchmarks it found that performance was particularly wanting. The elegant PowerPC was designed with floating point as a key consideration, and with these kind of tests, it should be on home turf.

As C't wryly notes, "In theory the PowerPC FPU with its 32 registers ought to have been superior to the x86 FPU with its antiquated stack structure and eight registers only."

The G4 scored between 147 and 187 on the floating point tests, while the Pentium III scored 297. Today's Pentium 4's double that figure, and as a result, today's PCs are four times as fast as Apple's professional line in some situations.

These are specialized tests, it's true, but they're the most closely watched by the kind of scientific and visualization users Apple has been trying to woo, and 'scientists' have been hardwired into Apple's mission statement for three years now. Credibility depends on success here.

But ever since Motorola decided to focus on embedded markets for its processors in Fall 1997, Steve Jobs has had to rely on one highly dubious bake off for his Pentium-trouncing demos. This squares a Mac Photoshop against a buggy, older version of Photoshop on Windows, and one that doesn't take advantage of the Screaming Sindy v2 instructions. Can you guess which one wins?

Apple has two performance headaches right now: the processor and the OS. Apple's director of core engineering Brett Halle last month promised us that OS X performance was a paramount concern, and to be fair, his division need take no blame. The BSD he inherited has the industry's best respected IP stack, for example.

Apple can draw on a few crumbs of comfort. The SPEC benchmarks disable a second CPU if present, and don't necessarily take advantage of the Altivec and Screaming Sindy dedicated FP instructions - that's down to the compiler. C't noted that it wasn't using the latest gcc compilers on the G4, but then again, it had shunned the latest Intel compilers too, which give a 20 per cent speed hike. And C't records that G4 performance is scaling better than the clock frequency, compared to the Pentium, with a 18.6:15.5 against the Pentium's 7:8. But given that that the Pentium is currently 2.2Ghz, and the G4 1Ghz, that will take a very long time indeed. (We've already disclosed how Apple's forthcoming chips use a larger 4MB cache, which is a much better way of getting there from here).

The problem is, one former Apple engineer told us, in serializing the twenty five year old BSD layer with the fifteen year old code of the extensions NeXT began to add in the mid 1980s.

Apple's attack parrots unfailingly point out that Mac OS X is "the most modern OS" you can use. In marketing terms, perhaps that's true. In technical terms, the integration point has been described to us as so crufty that no one wants to touch it. This is very old code indeed, and explains why you see the spinning disc cursor so often in OS X. Serialization is one of the toughest problems for an engineer to solve.

The more thoughtful majority of Mac users have real-world concerns about performance and have been expressing them to Apple, and to us, since the launch of X, and the latest SPEC results show that there's no place for self-delusion.

Over the past year we've consistently pointed out that Apple can make much of low-cost SMP systems, now that 's it got an OS capable of taking advantage of it. But we've also seen performance and latencies consistently fall short of equivalents in the PC world.

Apple needs a company-wide "performance task force" every much as Microsoft needs its "security task force", which was its belated response to persistent and chronic bad security practices. Any chance of a carefully leaked memo, Steve?®

Related Links

Apple's Gigahertz System and the SPEC Benchmark - C't [English version]
SPEC CPU2000 leader board
SPEC submissions for 2002

Related Story

New G4 roadmaps promise Apple harvest/A>

More about

TIP US OFF

Send us news


Other stories you might like