The Benchmark's Bunk – BAPCo a ‘Front’ for Intel

Or so rival claims

  • alert
  • submit to reddit

The two most widely used industry CPU benchmarks are effectively bunk, as they are under Intel's thumb, a rival maker claims.

Speaking at last week's Platform Conference held in San Jose, Randall Kennedy, the Director of Research for Competitive Systems Analysis, stated that BAPCo, the organization responsible for the most popular application level benchmark, SysMark 2000, was simply a "front" for the Satan Clara chip maker.

"It's my understanding that all of the other companies listed on BAPCo’s website have effectively just fallen by the wayside" in guiding the benchmark effort, he said.

Competitive Systems Analysis, Inc., also called "CSA Research," has developed a competitive benchmark suite marketed under the name Benchmark Studio.

According to Kennedy, Benchmark Studio was ironically borne from work he conducted under contract with Intel to demonstrate the superiority of the Pentium 4.

A demo application that preceded Benchmark Studio was used in a public presentation by Intel’s Pat Gelsinger last February, he said. Unfortunately these efforts eventually showed that dual-processing Pentium III systems, even when running at much slower clock speeds, easily outperformed even the fastest P4.

These results eventually led to a nasty falling out with the chip giant, according to Kennedy.

The current iteration of Benchmark Studio, version 1.0, shows a 733 MHz dual PIII system performing about twice as fast as a 1.5 GHz Pentium 4. Additionally, the benchmark also indicates that a 1.2 GHz AMD Athlon DDR system clobbered the same 1.5 GHz P4 system by about 15 per cent. The slowest system Kennedy discussed was one based on the 1 GHz Pentium III which was about 30 per cent slower than the P4.

Besides BAPCo's SysMark 2000, the second benchmark Kennedy singled out for being a Chipzilla chew toy was Ziff-Davis' Winstone suite. "Basically the benchmarks out there today say what Intel wants said" about computing performance, Kennedy claimed.

He explained that Benchmark Studio differs from other mainstream benchmarks by executing common tasks in the background. Built around out-of-process ActiveX COM objects which stress multitasking performance, Benchmark Studio gives a more realistic idea of system throughput, Kennedy claims.

Although Intel wants customers to believe that computing performance is like a stripped-down drag racer going from point A to point B, in real life computing is more like an SUV "running down the road with a load of gear in the back, a canoe on top and two screaming kids in the back shouting 'are we there yet?'" Kennedy stated.

You can take Benchmark Studio out for a spin by visiting CSA's website. Currently it's a free download.

And you can check out EETimes' take on the subject here. ®

Every benchmark tells a different story

Benchmarks - Itanic 32bit emulation is 'unusable'
P4 - Tom's Hardware has another rethink
P4: first Sandra 2001 benchmarks
Transmeta talks benchmarks...and 10 hour charges
Transmeta speed debate - damned lies and benchmarks?


Endpoint data privacy in the cloud is easier than you think
Innovations in encryption and storage resolve issues of data privacy and key requirements for companies to look for in a solution.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
Solving today's distributed Big Data backup challenges
Enable IT efficiency and allow a firm to access and reuse corporate information for competitive advantage, ultimately changing business outcomes.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.