Feeds

IBM gets handle on unstructured data

Almaden incarnates search and BI

Securing Web Applications Made Simple and Scalable

It is perhaps easy to assume that the notion of BI (business intelligence) for the masses - or 'DIYBI', as espoused here, is most likely to involve a sawn-off version of an existing BI tool - probably a mature one where the development costs have already been recovered.

In practice, this is somewhat less than likely, if only because most corporate BI tools are focused on working with structured data which is more found more extensively in large enterprises. It is already well understood that the majority of data in play in any business is unstructured and therefore not immediately well-suited to BI manipulations - and this is even more likely in the smaller businesses for which DIYBI might be attractive.

Developing technologies that can not only work with unstructured data but actually extract information from it of real value for the user is an important step along the way to DIYBI for the masses, but it also involves technology that goes well beyond what might be called a typical BI tool of today.

At one level, the basics of the 'DIY' toolset already exist in the form of the search engines such as Google, Yahoo! and MSN among many others. But search is a very minor part of BI, and in any environment where there is an embarrassment of unstructured data riches, can by itself be more of a hindrance than help.

The key here, according to Nelson Mattos, vice president of information & interaction research at IBM's Almaden Research Laboratories in California, is the ability to provide semantic analysis on both structured and unstructured data within a common environment.

"Users want to find whatever it is they want, regardless of where it is stored," he said. "They also want to continue to working with the tools they have and know, such as spreadsheets and Powerpoint presentations."

To that end, user interfaces are of equal importance to them, which means that search engine technologies have now become the UI of choice, says Mattos. "Everyone is already familiar with the search engine model - everyone can type a few words and get a result - and I want to use that paradigm in the context of a business intelligence environment."

IBM's research work is not intended to service the DIYBI market but it fits in with the notion of BI for the masses, for it is designed to support users running tasks associated with their jobs rather than be a tool for BI specialists. This is the target for Project Avatar, currently under development at Almaden.

Its object is to provide the tools that allow to users extract insight out of both structured and unstructured data. "This is a very broad area that requires management of structured and unstructured data and the use of traditional search technologies," Mattos said, "though that is not sufficient, because if you look at data across the enterprise there are huge amounts, so unless there is some intelligence to help find the insights we just overwhelm users with the amount of information."

Mattos sees search engines and BI coming together as the world of unstructured data is reeled in to the business need for intelligence. “At the moment 80-85 per cent of the data stored on computers is unstructured,” he said, “so it would be good to have a common framework that will allow users to analyse a record or historical data to identify problems, issues or trends – analysing structured and unstructured data together.”

The paradigms used to interface with that combined framework may look like a search engine but they will be different. The text related search technologies have been, until recently, solely built on keywords not semantics. There have been some sophisticated algorithms developed that can look at keywords in a context, recognising company names, technical components and the like. But they were developed in a proprietary fashion and that, Mattos suggests, is why they have never taken off.

Bridging the IT gap between rising business demands and ageing tools

More from The Register

next story
NO MORE ALL CAPS and other pleasures of Visual Studio 14
Unpicking a packed preview that breaks down ASP.NET
DARPA-derived secure microkernel goes open source tomorrow
Hacker-repelling, drone-protecting code will soon be yours to tweak as you see fit
Cheer up, Nokia fans. It can start making mobes again in 18 months
The real winner of the Nokia sale is *drumroll* ... Nokia
Put down that Oracle database patch: It could cost $23,000 per CPU
On-by-default INMEMORY tech a boon for developers ... as long as they can afford it
Google shows off new Chrome OS look
Athena springs full-grown from Chromium project's head
Apple: We'll unleash OS X Yosemite beta on the MASSES on 24 July
Starting today, regular fanbois will be guinea pigs, it tells Reg
HIDDEN packet sniffer spy tech in MILLIONS of iPhones, iPads – expert
Don't panic though – Apple's backdoor is not wide open to all, guru tells us
prev story

Whitepapers

Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
Seven Steps to Software Security
Seven practical steps you can begin to take today to secure your applications and prevent the damages a successful cyber-attack can cause.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.