Original URL: http://www.theregister.co.uk/2006/07/30/hate_surveys/

I hate surveys!

But I wouldn't want to go over the top about it, right?

By David Norfolk

Posted in Developer, 30th July 2006 21:13 GMT

According to a Microsoft press release, based on a survey by Vanson Bourne of 200 UK businesses and commissioned by Microsoft Ltd (and quoted, somewhat reluctantly, by Martin Banks here):

“Alarmingly, in 66 per cent of UK businesses using BI tools, organisations’ business intelligence resides with their employees. And only 2 per cent of organisations score their business IQ above 90 per cent with as high as 83 per cent rating it below 75 per cent”.

But take out adjectives like “alarmingly”, and ask what a “Business IQ” of 75 per cent really means, and it seems to me that you can’t (or shouldn’t) read anything much at all into this survey – or most others these days.

I was tempted to interpret this report as showing that surveys today are usually a pointless PR exercise <g>. Nevertheless, I can't really go on to infer that either or both the people doing the work or Microsoft's BI team don't know what BI is (although I suppose that that is always a possibility) because there just isn't enough information available for such a potentially useful conclusion.

Let's look at some of the results in another way:

Some 43 per cent of responders use SQL Server 2000, 21 per cent use SQL Server 2005 and 13 per cent use SQL Server 7 (v7 was the first SQL Server version that was a total rewrite from the original Sybase product). I've no idea how much overlap there is between these groups (I don't think the survey asked), but can I conclude that SQL Server 2005 (which is the latest release, despite the name, and the first one to really compete with the Big Boys in BI, allegedly), seems to have some way to go to oust 2000? Is this a problem for Microsoft and does it imply that MS is simply mopping up the Mickey Mouse BI opportunities, in the main, for which earlier releases of SQL Server are adequate?

Now, despite "only" (my adjective) 46 per cent of companies using BI tools, 66 per cent of responders thought that the "organisation's intelligence" resided with its people and "an impressive" (me again) 58 per cent rated their company's intelligence at analysing info as "OK". In fact, an "extremely significant" (sorry, me once more) 91 per cent rated themselves as OK, Good, or Outstanding at this. Clear evidence, one might argue, that BI tools are often unnecessary in practice.

Well that was fun <g>, but the point is that most marketing-led surveys are information-free areas and can be used to support any position you want them to. And that sample size – 200 – was it large enough for any sort of statistical reliability, I wonder?

Martin Banks, in fact, thinks that someone behind this survey may be confusing staff tacit knowledge – which is a knowledge management issue – with BI. In small companies, there are probably cultural reasons for some `BI-applicable’ information to be retained in a brain rather than a database, he thinks. And it seems a bit of a jump, to him, to go from the majority of an organisation’s intelligence residing with people to the need for a BI system – as the press release which came with this survey does.

Fair enough, although I think that distinguishing BI from Decision Support from Knowledge Management etc is, ultimately, pointless. You need processes (both manual and automated) for your operation, driven by policies. These (and the requirements of internal and external stakeholders such as regulators for governance information and transparency) drive what information needs to be made available to decision makers - then you can choose tools that can deliver this.

It could indeed be argued that, as Martin pointed out to me, for many smaller businesses, utilizing the human brain is by far the best policy option for most business intelligence's requirements. BI for the masses sounds like a good idea, he says – it probably will be in time – but it does sometimes seem like ISVs trying to sell users something they don’t need to solve a problem they don’t have, just in case some other vendor sells it to them first.

But, back to where I came in, please let’s have fewer marketing efforts disguised as surveys which are “all sound and fury, signifying nothing”. Pretty please?