The Business Intelligence (BI) scandal: why pay more to get less?

Comment The eye-catching headline above recently appeared in a press release from “business data specialists ICS". What!? A scandal in the moral world of BI? Surely not.

It turns out that the burden of ICS's song is that it has a product that "puts information in the hands of the user rather than confined to business analysts away from the coalface". And the scandal is that "traditional BI systems are clunky and over-priced [and are] leaving the real business decision makers out in the cold".

Initially, it's a compelling argument. I have to agree that within many of the companies I see, BI use is confined to higher managers - those traditional "decision makers". BI often does fail to get down as far as those at the coalface.

But is this really a scandal? Is it actually appropriate for all companies to deploy BI to employees at all levels? That would be a No. There are, sadly, some coalface jobs where access to corporate BI systems has very little relevance. I know; when I was a student, I did plenty of them. I remember long summer days devoted to the taxing pursuit of removing chicken excrement from nesting boxes. The company wisely chose to issue me with a power hose rather than a BI system.

I also happily concede that there are plenty of coalface jobs where access to a BI system could make a huge difference. BI systems can detect real-time credit fraud; they can warn sales staff of potentially abusive or dangerous customers, it can alert shelf stackers to changing shopping requirements and so on. However, I also believe that deployment at this level is far more likely to be hindered by human/social reasons rather than technological ones.

For a start, many companies actively want the coalface employees to be cogs in the machine. They truly and actively do not want their waitpersons to demonstrate initiative. It is much safer, and the cogs’ performance is much more predictable, if they all simply stick to the script. I'm not defending this as a position; I’m simply explaining why the idea of introducing BI at the cog level is essentially ruled out, in some cases, by company policy.

OK, let's move higher up the food chain. At some point, even in the most cog-oriented organisation, we find a level at which managers are expected to make sane and sensible decisions based on the available information. Clearly such people will benefit from access to a BI system that helps them to extract useful information from the mass of data that confronts them. Technologically there is no reason why they cannot be given access, but someone has to decide to give it to them.

Why would anyone higher up the chain object? Well, for a start, good quality information might allow the lower orders to questions the decisions of their higher-ups. It might enable the great unwashed to see that some decisions (promotions for example) are based on favour rather than fact. The bottom line is that knowledge is power (as Sir Francis Bacon knew way, way before Business acquitted even a hint of Intelligence). If you hold the knowledge, and your underlings don't, then your power base is safe.

Ultimately, there are many reasons why BI is often restricted to the upper levels of management – simply providing a new tool will not, automatically, fix the problem.