Feeds

Too many cooks spoil the data warehouse broth

Cut your staffing costs

Secure remote control for conventional and virtual desktops

In many organisations the costs of data warehousing are expressed on the balance sheet as the hardware, software and administration costs that support the operation.

In practice, many hidden costs are associated with the data warehouse with the structure of an organisation.

So says Mark Thomas, IBM’s Infosphere and data warehousing specialist. “The cost of a complex data warehouse is more than that expressed on the balance sheet,” he says.

“It is more than the hardware, the software and the people that work directly with the data warehouse. There are business units outside the IT organisation that almost certainly have people working with the data warehouse as well.”

In, out, shake it all about

These staff may be used to get data in to and out of the data warehouse for analytics or reporting. Thomas cites a recent project with a large bank where it was discovered that 106 database administration staff were associated with the data warehouse in IT and across lines of business.

“These were staff who were accessing and manipulating the data – not productive work but getting data in and out, and keeping the surrounding business intelligence environment going,” he says.

“There’s a huge amount of cost involved in keeping data warehouses going, just in moving data as well as indexing and partitioning and tuning. There can be a lot of people involved just to make sure the data is being jammed in and the associated processing work goes on.

“It’s a very inefficient use of manpower, just keeping infrastructure working rather than creating new insight or driving business analytics for competitive advantage.”

Companies are used to living with this state of affairs. Thomas says many of them recognise the scale of the costs and inefficiencies associated with the data warehouse, but because these costs tend to be apportioned around the organisation there is often little impetus to do anything about it.

He cites the case of another bank where the department that was the main user of the data warehouse was accessing it for free. No one had thought to charge the unit for it, and the department managers certainly weren’t going to bring this oversight to anyone’s attention.

Thomas adds that often the internal cost charges associated with moving data in or out of the data warehouse can be prohibitive, preventing analysts from trying new things.

Handy appliance

IBM’s response to this can be summed up in two somewhat ungainly words: “pre-integration” and “pre-configuration”.

With IBM’s big data suite an awful lot of the costs usually associated with data warehousing are removed, the company claims. The new breed of appliances championed by IBM includes technologies from Netezza, the data warehousing appliance company.

The vendor claims the result is a highly flexible and robust warehouse which merges the traditional and big data approaches, coping with both structured and unstructured data. It hugely simplifies the database environment because all the components are optimised to work together.

“Resource can be used to get projects out of the door”

“You have a far more manageable environment and scarce IT resources can be better used to create value,” says Thomas.

“It’s a simple and manageable data warehouse which changes the cost paradigm from a staffing perspective. Resource can be used in a more productive manner – to meet business requirements and get projects out of the door, rather than just keeping the machine going.”

Scarce resources are deployed more effectively and implementation costs are also much lower, IBM claims.

Change the tuning

“So much comes pre-integrated and pre-configured,” says Thomas. “Very often implementations are about getting data into the right applications. That remains, but there is no configuration so implementation is very simple and quick.

“We have a client who has had a Netezza appliance for about a year, and as yet hasn’t needed any training. It is all so simple and it can’t be altered that much.”

Thomas concludes that a traditional data warehouse is a hugely costly beast because it requires an infrastructure that is too complex, requires too many people to maintain it and too much tuning to meet the ever changing needs of the business, particularly at a time when IT is under ever greater pressure to reduce costs.

“Netezza and our associated information management tools offer massive performance out of the box, without the traditional need for tuning, indexing or partitioning,” he says.

“It doesn’t require a team of expensive specialists and significantly speeds up time to market of the deliverables from the data warehouse.” ®

Next gen security for virtualised datacentres

More from The Register

next story
6 Obvious Reasons Why Facebook Will Ban This Article (Thank God)
Clampdown on clickbait ... and El Reg is OK with this
No, thank you. I will not code for the Caliphate
Some assignments, even the Bongster decline must
Caught red-handed: UK cops, PCSOs, specials behaving badly… on social media
No Mr Fuzz, don't ask a crime victim to be your pal on Facebook
Barnes & Noble: Swallow a Samsung Nook tablet, please ... pretty please
Novelslab finally on sale with ($199 - $20) price tag
Ballmer leaves Microsoft board to spend more time with his b-balls
From Clippy to Clippers: Hi, I see you're running an NBA team now ...
Banking apps: Handy, can grab all your money... and RIDDLED with coding flaws
Yep, that one place you'd hoped you wouldn't find 'em
Video of US journalist 'beheading' pulled from social media
Yanked footage featured British-accented attacker and US journo James Foley
Call of Duty daddy considers launching own movie studio
Activision Blizzard might like quality control of a CoD film
Primetime precrime? Minority Report TV series 'being developed'
I have to know. I have to find out what happened to my life
prev story

Whitepapers

A new approach to endpoint data protection
What is the best way to ensure comprehensive visibility, management, and control of information on both company-owned and employee-owned devices?
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Maximize storage efficiency across the enterprise
The HP StoreOnce backup solution offers highly flexible, centrally managed, and highly efficient data protection for any enterprise.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Next gen security for virtualised datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.