Have a reality check, you Itanium lover

Benchmark journey

  • alert
  • submit to reddit

Gartner critical capabilities for enterprise endpoint backup

Letter We don't typically spend a lot of time debating benchmarks here at Vulture Central, primarily because of all the work vendors pour into tuning their systems to perform well. You often find servers with unusual features such as excessive memory or you see tweaks to processors that would not likely occur in the real world.

One Reg reader took the time to have a close look at some of the most popular benchmarks and show how this vendor-tweaking plays out - at least in the case of Intel's Itanium processor.

In your recent article "Analyst sees St. Fister in Itanium wafer," you included the statements "in 2003, Intel managed to capture most of the leading performance benchmarks with EPIC/Itanium architecture" and "Clabby rightly points out that Intel has destroyed the competition on most benchmarks with IBM's Power processor the only rival even close to keeping pace."

This primarily proves that if one sprays around enough Kool-Aid it will touch even the lips of those who don't actually care for it: Intel has had considerable success in doing this for Itanic, and HP perhaps even more so.

Let's look at the record.

SPECint (the processor metric of most interest to most server users): By virtue of its monstrous on-chip L3 cache (rather than anything intrinsic to its EPIC architecture), Itanic briefly grabbed partial SPECint leadership (in base but not peak scores) over its RISC competition when the McKinley product was nominally launched in July, 2002. But its base scores still fell significantly behind those of its P4/Xeon sibling and AMD's Athlon, which without question also qualify as server competition, so no brass ring there at all. When Madison was launched in June, 2003 it did somewhat better: doubling the size of the already-monstrous L3 cache gave it at least a marginal SPECint win over *all* competition for a while.

But again there are qualifications to note: 1) this SPECint leadership was only obtainable using HP's zx1 chipset and its HP-UX compiler (performance in other configurations only matched that of IBM's new POWER4+, and fell short of the fastest P4/Xeons and new AMD64 products), and 2) AMD64 and P4 have since forged ahead of even those leading HP platform results. So Itanic not only failed to 'destroy' the competition in SPECint, it isn't currently even a leader at all, and from published roadmaps seems unlikely to retake the lead any time soon.

SPECfp (the processor metric of most interest to many HPTC users): This is the *one* area where Itanic can legitimately boast of a clear lead, and can actually credit that lead to its EPIC architecture rather than to other chip or system features. However, there's a good reason why HPTC users haven't jumped on the Itanic bandwagon to the exclusion of all other platforms: they're also sensitive to things like performance per Watt and performance per dollar, and Itanic does not lead in either of these.

TPC-C (a server metric of significant interest to database users, at least in its unclustered form): Itanic's performance here (again) in no way 'destroys' - or even clearly leads - its competition.

1) HP's zx1-based 4-processor platform has a clear TPC-C lead at that node size, but not due to any EPIC features: 4-way Xeon systems support only 1/3 the on-chip cache, 1/3 the maximum RAM, and 1/3 the aggregate FSB bandwidth to work with, IBM hasn't submitted any top-of-the-line systems at this system size, and the cost of setting up a full-bore TPC-C test seems to have prevented any serious AMD64 submissions since the initial one nearly a year ago (its submission date was changed last summer due to insignificant revisions).

2) HP's 64-processor Superdome indeed posts the top TPC-C score, but IBM's POWER4+ p690 attains 76% of that score using only half as many processors (and only 1/4 as many processor chips, since each is dual-core - you mention future Itanic multi-core chips later in your article, so this might be of interest): on a per-processor basis POWER4+ delivers 33% higher TPC-C performance, and on a per-processor-chip basis it delivers 167% higher TPC-C performance (these numbers based on comparison with Itanic's best 32-processor TPC-C result to avoid apples-to-oranges issues: relative per-processor performance in the 64-processor Superdome system listed is even worse).

SAP SD 2-tier (another server metric of significant interest to database users): IBM simply hasn't bothered to submit top-of-the-line configurations here, but their previous-generation 1.3 GHz POWER4 32-processor p690 system beats the best current-generation top-of-the-line 1.5 GHz Itanic 32-processor score (so, for that matter, does the 32-processor 1.15 GHz Alpha GS1280 system - again using previous-generation chip technology), and the score obtained by IBM's mid-range 1.45 GHz p650 8-way server suggests that its high-end model would approximately equal the top-of-the-line Itanic 2 at the 8-processor node size - so no 'destruction' (or even clear leadership at all) by Itanic here, either.

Oracle Applications Standard Benchmark (yet another significant database benchmark): No direct comparison with POWER4+ is available, and AMD64 systems haven't been submitted here. But the 16-processor POWER4+ score is over 3.4 times the 4-processor Itanic2 score, so given the significantly less than linear scaling up from 4 processors that HP's systems have demonstrated in other database-related areas POWER4+ seems likely to hold at the very least per-processor equality here.

SPECweb99_SSL (an important web-serving benchmark): HP initially seemed to have chosen to submit a SPECweb99_SSL Itanic test instead of one using its more popular SPECweb99 sibling because there was no serious competition there, but times have changed: in the hotly-contested 4-processor node size AMD64 now holds a modest lead, with Itanic2 and POWER4+ in a tie for second.

SPECjbb2000 (an important enterprise server benchmark): Itanic holds a slim (under 3%) lead over AMD64 at the 4-processor node size and another slim (under 4%) lead over POWER4+ at the 32-processor node size - hardly 'destroying' the competition, once again. And while Superdome's support for up to 64 processors lets Itanic exceed the p690's absolute headroom, the top score in the benchmark is held by a 112-processor Fujitsu SPARC64 (gasp!) system.

So there you have Itanic's benchmark 'dominance', using about the same set of benchmarks that HP uses to try to prove it (but without the apples-to-oranges spin that they add): one clear win (SPECfp), three approximate ties (OASB - maybe, SPECweb99_SSL, and SPECjbb2000), and three (TPC-C, SPECint, and SAP SD 2-tier) where Itanic is clearly behind at some system sizes and/or apparently mostly benefiting from the lack of competing
submissions elsewhere).

POWER4+ is not some struggling also-ran as your quotes above suggest: on average, it's measurably superior to Itanic 2 (and should increase that lead significantly when POWER5 appears this year), and where AMD64 has shown up to compete it's on average at least Itanic 2's equal (and with comparable benchmarketing effort would likely be shown to be its superior). Alpha would be putting Itanic to shame if Compaq hadn't decided to start de-emphasizing Alpha development over 4 years ago, and PA-RISC might as well had HP not made a similar decision even longer ago - but Itanic still has plenty of competition left (as noted above, even SPARC64 can't be ignored), and another serious threat may develop if Intel decides that it has to counter AMD64 with an x86 64-bit extension of its own.

Bill Todd ®

Secure remote control for conventional and virtual desktops

More from The Register

next story
The Return of BSOD: Does ANYONE trust Microsoft patches?
Sysadmins, you're either fighting fires or seen as incompetents now
Microsoft: Azure isn't ready for biz-critical apps … yet
Microsoft will move its own IT to the cloud to avoid $200m server bill
Oracle reveals 32-core, 10 BEEELLION-transistor SPARC M7
New chip scales to 1024 cores, 8192 threads 64 TB RAM, at speeds over 3.6GHz
US regulators OK sale of IBM's x86 server biz to Lenovo
Now all that remains is for gov't offices to ban the boxes
Object storage bods Exablox: RAID is dead, baby. RAID is dead
Bring your own disks to its object appliances
Nimble's latest mutants GORGE themselves on unlucky forerunners
Crossing Sandy Bridges without stopping for breath
prev story


Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Top 10 endpoint backup mistakes
Avoid the ten endpoint backup mistakes to ensure that your critical corporate data is protected and end user productivity is improved.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
Rethinking backup and recovery in the modern data center
Combining intelligence, operational analytics, and automation to enable efficient, data-driven IT organizations using the HP ABR approach.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.