Feeds

Too many cooks spoil the data warehouse broth

Cut your staffing costs

Security for virtualized datacentres

In many organisations the costs of data warehousing are expressed on the balance sheet as the hardware, software and administration costs that support the operation.

In practice, many hidden costs are associated with the data warehouse with the structure of an organisation.

So says Mark Thomas, IBM’s Infosphere and data warehousing specialist. “The cost of a complex data warehouse is more than that expressed on the balance sheet,” he says.

“It is more than the hardware, the software and the people that work directly with the data warehouse. There are business units outside the IT organisation that almost certainly have people working with the data warehouse as well.”

In, out, shake it all about

These staff may be used to get data in to and out of the data warehouse for analytics or reporting. Thomas cites a recent project with a large bank where it was discovered that 106 database administration staff were associated with the data warehouse in IT and across lines of business.

“These were staff who were accessing and manipulating the data – not productive work but getting data in and out, and keeping the surrounding business intelligence environment going,” he says.

“There’s a huge amount of cost involved in keeping data warehouses going, just in moving data as well as indexing and partitioning and tuning. There can be a lot of people involved just to make sure the data is being jammed in and the associated processing work goes on.

“It’s a very inefficient use of manpower, just keeping infrastructure working rather than creating new insight or driving business analytics for competitive advantage.”

Companies are used to living with this state of affairs. Thomas says many of them recognise the scale of the costs and inefficiencies associated with the data warehouse, but because these costs tend to be apportioned around the organisation there is often little impetus to do anything about it.

He cites the case of another bank where the department that was the main user of the data warehouse was accessing it for free. No one had thought to charge the unit for it, and the department managers certainly weren’t going to bring this oversight to anyone’s attention.

Thomas adds that often the internal cost charges associated with moving data in or out of the data warehouse can be prohibitive, preventing analysts from trying new things.

Handy appliance

IBM’s response to this can be summed up in two somewhat ungainly words: “pre-integration” and “pre-configuration”.

With IBM’s big data suite an awful lot of the costs usually associated with data warehousing are removed, the company claims. The new breed of appliances championed by IBM includes technologies from Netezza, the data warehousing appliance company.

The vendor claims the result is a highly flexible and robust warehouse which merges the traditional and big data approaches, coping with both structured and unstructured data. It hugely simplifies the database environment because all the components are optimised to work together.

“Resource can be used to get projects out of the door”

“You have a far more manageable environment and scarce IT resources can be better used to create value,” says Thomas.

“It’s a simple and manageable data warehouse which changes the cost paradigm from a staffing perspective. Resource can be used in a more productive manner – to meet business requirements and get projects out of the door, rather than just keeping the machine going.”

Scarce resources are deployed more effectively and implementation costs are also much lower, IBM claims.

Change the tuning

“So much comes pre-integrated and pre-configured,” says Thomas. “Very often implementations are about getting data into the right applications. That remains, but there is no configuration so implementation is very simple and quick.

“We have a client who has had a Netezza appliance for about a year, and as yet hasn’t needed any training. It is all so simple and it can’t be altered that much.”

Thomas concludes that a traditional data warehouse is a hugely costly beast because it requires an infrastructure that is too complex, requires too many people to maintain it and too much tuning to meet the ever changing needs of the business, particularly at a time when IT is under ever greater pressure to reduce costs.

“Netezza and our associated information management tools offer massive performance out of the box, without the traditional need for tuning, indexing or partitioning,” he says.

“It doesn’t require a team of expensive specialists and significantly speeds up time to market of the deliverables from the data warehouse.” ®

Security and trust: The backbone of doing business over the internet

More from The Register

next story
Phones 4u slips into administration after EE cuts ties with Brit mobe retailer
More than 5,500 jobs could be axed if rescue mission fails
JINGS! Microsoft Bing called Scots indyref RIGHT!
Redmond sporran metrics get one in the ten ring
Driving with an Apple Watch could land you with a £100 FINE
Bad news for tech-addicted fanbois behind the wheel
Phones 4u website DIES as wounded mobe retailer struggles to stay above water
Founder blames 'ruthless network partners' for implosion
Sony says year's losses will be FOUR TIMES DEEPER than thought
Losses of more than $2 BILLION loom over troubled Japanese corp
Radio hams can encrypt, in emergencies, says Ofcom
Consultation promises new spectrum and hints at relaxed licence conditions
Why Oracle CEO Larry Ellison had to go ... Except he hasn't
Silicon Valley's veteran seadog in piratical Putin impression
Big Content Australia just blew a big hole in its credibility
AHEDA's research on average content prices did not expose methodology, so appears less than rigourous
prev story

Whitepapers

Secure remote control for conventional and virtual desktops
Balancing user privacy and privileged access, in accordance with compliance frameworks and legislation. Evaluating any potential remote control choice.
WIN a very cool portable ZX Spectrum
Win a one-off portable Spectrum built by legendary hardware hacker Ben Heck
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
The next step in data security
With recent increased privacy concerns and computers becoming more powerful, the chance of hackers being able to crack smaller-sized RSA keys increases.