Feeds

IBM gets handle on unstructured data

Almaden incarnates search and BI

New hybrid storage solutions

It is perhaps easy to assume that the notion of BI (business intelligence) for the masses - or 'DIYBI', as espoused here, is most likely to involve a sawn-off version of an existing BI tool - probably a mature one where the development costs have already been recovered.

In practice, this is somewhat less than likely, if only because most corporate BI tools are focused on working with structured data which is more found more extensively in large enterprises. It is already well understood that the majority of data in play in any business is unstructured and therefore not immediately well-suited to BI manipulations - and this is even more likely in the smaller businesses for which DIYBI might be attractive.

Developing technologies that can not only work with unstructured data but actually extract information from it of real value for the user is an important step along the way to DIYBI for the masses, but it also involves technology that goes well beyond what might be called a typical BI tool of today.

At one level, the basics of the 'DIY' toolset already exist in the form of the search engines such as Google, Yahoo! and MSN among many others. But search is a very minor part of BI, and in any environment where there is an embarrassment of unstructured data riches, can by itself be more of a hindrance than help.

The key here, according to Nelson Mattos, vice president of information & interaction research at IBM's Almaden Research Laboratories in California, is the ability to provide semantic analysis on both structured and unstructured data within a common environment.

"Users want to find whatever it is they want, regardless of where it is stored," he said. "They also want to continue to working with the tools they have and know, such as spreadsheets and Powerpoint presentations."

To that end, user interfaces are of equal importance to them, which means that search engine technologies have now become the UI of choice, says Mattos. "Everyone is already familiar with the search engine model - everyone can type a few words and get a result - and I want to use that paradigm in the context of a business intelligence environment."

IBM's research work is not intended to service the DIYBI market but it fits in with the notion of BI for the masses, for it is designed to support users running tasks associated with their jobs rather than be a tool for BI specialists. This is the target for Project Avatar, currently under development at Almaden.

Its object is to provide the tools that allow to users extract insight out of both structured and unstructured data. "This is a very broad area that requires management of structured and unstructured data and the use of traditional search technologies," Mattos said, "though that is not sufficient, because if you look at data across the enterprise there are huge amounts, so unless there is some intelligence to help find the insights we just overwhelm users with the amount of information."

Mattos sees search engines and BI coming together as the world of unstructured data is reeled in to the business need for intelligence. “At the moment 80-85 per cent of the data stored on computers is unstructured,” he said, “so it would be good to have a common framework that will allow users to analyse a record or historical data to identify problems, issues or trends – analysing structured and unstructured data together.”

The paradigms used to interface with that combined framework may look like a search engine but they will be different. The text related search technologies have been, until recently, solely built on keywords not semantics. There have been some sophisticated algorithms developed that can look at keywords in a context, recognising company names, technical components and the like. But they were developed in a proprietary fashion and that, Mattos suggests, is why they have never taken off.

Secure remote control for conventional and virtual desktops

More from The Register

next story
Not appy with your Chromebook? Well now it can run Android apps
Google offers beta of tricky OS-inside-OS tech
Keep that consumer browser tat away from our software says Oracle
Big Red decides it will only support Firefox's Extended Support Releases
Greater dev access to iOS 8 will put us AT RISK from HACKERS
Knocking holes in Apple's walled garden could backfire, says securo-chap
WordPress 4.0 is here, complete with one-click upgrade process
Don't relax yet, sysadmins, there's still a chance for some big messes here
NHS grows a NoSQL backbone and rips out its Oracle Spine
Open source? In the government? Ha ha! What, wait ...?
Google extends app refund window to two hours
You now have 120 minutes to finish that game instead of 15
Intel: Hey, enterprises, drop everything and DO HADOOP
Big Data analytics projected to run on more servers than any other app
prev story

Whitepapers

Secure remote control for conventional and virtual desktops
Balancing user privacy and privileged access, in accordance with compliance frameworks and legislation. Evaluating any potential remote control choice.
Intelligent flash storage arrays
Tegile Intelligent Storage Arrays with IntelliFlash helps IT boost storage utilization and effciency while delivering unmatched storage savings and performance.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.
Providing a secure and efficient Helpdesk
A single remote control platform for user support is be key to providing an efficient helpdesk. Retain full control over the way in which screen and keystroke data is transmitted.