IBM, HP annoy server buyers less than Dell, Sun
x64 FAIL meter
The annual x64 server customer survey, put together by Gabriel Consulting Group, is out this week, and the top four server makers in this subset of the server space - Hewlett-Packard, Dell, IBM, and Sun Microsystems - have been given their report cards concerning how well they put together machines, deliver performance and features, and otherwise satisfy their customers.
The x64 survey is knocked together once a year by Dan Olds, the principal researcher at the market research firm and a former server marketer who has had stints at Sequent, Sun Microsystems, and IBM before striking out on his own. Olds also does a Unix market survey once a year, and both surveys provide some insight not only into what server buyers are doing with their iron, but also the issues that they are thinking about as they acquire and deploy boxes. About 80 per cent of the people that Olds surveyed in the first half of 2008 came from the United States or Europe.
There were a total of 187 respondents to this x64 survey. Small and mid-sized businesses (SMBs) were well represented in the survey, making up 43 per cent of total participants. This survey also had a reasonable number of very large enterprise participants, at a little over 11 per cent. About a third of the shops who participated in the survey this time around have fewer than 20 servers, but a little more than a third have between 20 and 50 x64 machines. Another third have more than 50 boxes, and quite a few (9 per cent) have more than 250 machines. This seems like a fairly representative cross section of the x64 base.
While the x64 user survey focuses on many different aspects of the gear and their use, for simplicity's sake this article is going to stick with the vendor face-off portion of the survey, which pits the server makers against each other. (I'll do a follow-up looking at operating system, virtualization, power and cooling, and other issues.) To sum up the data succinctly, x64 iron from HP and IBM is well regarded among those polled, followed by Sun and with Dell brining up the rear. Interestingly, on many of the metrics that Gabriel Consulting is tracking in the survey, all four vendors are falling. This may mean expectations are rising, and it might mean server makers are cutting corners.
After counting up the responses to questions, Olds normalizes his data into something called the Vendor Preference Index (VPI). The VPI compares the number of "votes" a vendor gets to the number of respondents who said they have standardized on that vendor. As Olds explains it, the VPI is designed to keep a particular vendor's installed base from skewing the customer ratings, so the numbers provided by survey takers are divided by the number of people who say they have standardized on a particular vendor. So, if 100 people standardize on, for example, HP's x64 servers, and a survey gets 100 positive responses, then this is par for the course. Anything below 100 is not so good, and anything above 100 is exceptional. (That would mean that customers standardizing on other iron had rated HP better than their own vendor.) The idea behind VPI is to understand how IT shops rate server brands against each other, not just measure how prevalent each vendor is in their data centers and what they think about the dominant boxes in their sites.
In the x64 market, bang for the buck is probably the most important metric. And it is here where the numbers are dropping over the past three surveys with the exception of a 2008 wiggle from IBM and Dell. The important thing is that the overall trend is down. Check it out:
As you can see, the gaps between the four vendors were pretty tight in the 2006 survey, but VPI scores are down from these levels in the 2008 and the gaps between vendors are a little wider than they were three years ago.
While HP and Dell are the volume leaders in the global x64 market and IBM is a distant third with Sun being an even more distant fourth, the shipment rates each quarter from the server makers are as much a reflection of the inertia in their installed bases, the relative strengths and weaknesses of their channels, and the new technologies they bring to market each year. It takes a long, long time to change momentum in the server space. Sun's strategy with its "Galaxy" x64 server line has been focused on providing density at a lower price, but the numbers that Gabriel Consulting has gathered show that Sun still has a long way to go.
The Performance Gap
"IBM is the perceived leader in performance, and HP leads in systems management," explains Olds. "Dell is seen as a provider of boxes with little to no differentiation. And while some Sun customers are zealous about Sun iron, the company has not made up the ground between itself and HP and IBM."
The difference in the raw performance across these four vendors that customers say their suppliers are delivering is widening. And as the numbers below show, there is quite a bit of differentiation among these four vendors, even given the commodity nature of the x64 server these days.
The differences in the VPI rankings for the four major x64 server vendors is similarly striking, and shows that perceptions among customers using a mix of different server products are hard to change, particularly when it comes to system reliability:
Now, Olds knows the difference between perception and reality, and says that the feature set between various x64 server makers is not that different. Dell systems are not that far behind. But he does believe that some vendors do a better job at stressing performance or availability features, and do a better job of marketing these features to customers. The marketing message is slippery, and if it didn't work, vendors wouldn't do it. For instance, IBM has pretty much owned the raw performance metric in the x64 server survey for the past three years. But when you ask customers about the observed performance of the machines in their data centers, IBM and HP have scored a VPI at 100 or higher each time, and Sun has done better than its raw performance metrics above show. Dell also did better, averaging around a 70 VPI across the three surveys. The gap between raw and observed performance is a measure of sorts of the effectiveness of benchmarketing.
If there is another area where HP and IBM are doing better than Dell and Sun, it is in systems management. In the 2008 survey, HP scored a 109.5 VPI on its system management tools (that would be Insight Manager and related tools) and IBM fell quite a bit, from a 114.7 VPI in 2007 with its Systems Director and related tools to a 96.65 VPI in 2008. But Dell rated only a 57.9 VPI in 2008 (up more than five points), and Sun, while up nearly 12 points, is still only at an 88.2 VPI with its systems management tools as rated by those polled by Gabriel Consulting.
Another issue that bothers customers is technology roadmaps. When they invest in a product line, they want to have a sense of what the future product line is going to look like. And as you might expect, HP and IBM seem to be doing the best job here, too. Take a gander at the data for the past three years:
Of course, in the case of the x64 server space, the roadmaps are driven mostly by chip makers Intel and Advanced Micro Devices, and when either of these two sneezes, the server vendors get the cold.
As in past surveys, most of the shops polled have x64 iron from multiple vendors. In the 2008 survey, 22 per cent of the shops polled had three different sever vendors, 37 per cent had four, and 22 per cent had five. Only 7 per cent reported having only one vendor, in fact. And it doesn't look like anyone is getting ready to pick one vendor and buy all their gear from that company, either.
"Since we have had two vendors back in the x86 days, people have been talking about standardizing on one set of products," says Olds. "And vendors have been talking about this, too. But it just ain't happening. Customers love a horse race when it comes to x64 servers." And that is particularly good news for the top four x64 server makers and their competitors, who also want to get a piece of the action. ®