Feeds

IBM gets handle on unstructured data

Almaden incarnates search and BI

Internet Security Threat Report 2014

It is perhaps easy to assume that the notion of BI (business intelligence) for the masses - or 'DIYBI', as espoused here, is most likely to involve a sawn-off version of an existing BI tool - probably a mature one where the development costs have already been recovered.

In practice, this is somewhat less than likely, if only because most corporate BI tools are focused on working with structured data which is more found more extensively in large enterprises. It is already well understood that the majority of data in play in any business is unstructured and therefore not immediately well-suited to BI manipulations - and this is even more likely in the smaller businesses for which DIYBI might be attractive.

Developing technologies that can not only work with unstructured data but actually extract information from it of real value for the user is an important step along the way to DIYBI for the masses, but it also involves technology that goes well beyond what might be called a typical BI tool of today.

At one level, the basics of the 'DIY' toolset already exist in the form of the search engines such as Google, Yahoo! and MSN among many others. But search is a very minor part of BI, and in any environment where there is an embarrassment of unstructured data riches, can by itself be more of a hindrance than help.

The key here, according to Nelson Mattos, vice president of information & interaction research at IBM's Almaden Research Laboratories in California, is the ability to provide semantic analysis on both structured and unstructured data within a common environment.

"Users want to find whatever it is they want, regardless of where it is stored," he said. "They also want to continue to working with the tools they have and know, such as spreadsheets and Powerpoint presentations."

To that end, user interfaces are of equal importance to them, which means that search engine technologies have now become the UI of choice, says Mattos. "Everyone is already familiar with the search engine model - everyone can type a few words and get a result - and I want to use that paradigm in the context of a business intelligence environment."

IBM's research work is not intended to service the DIYBI market but it fits in with the notion of BI for the masses, for it is designed to support users running tasks associated with their jobs rather than be a tool for BI specialists. This is the target for Project Avatar, currently under development at Almaden.

Its object is to provide the tools that allow to users extract insight out of both structured and unstructured data. "This is a very broad area that requires management of structured and unstructured data and the use of traditional search technologies," Mattos said, "though that is not sufficient, because if you look at data across the enterprise there are huge amounts, so unless there is some intelligence to help find the insights we just overwhelm users with the amount of information."

Mattos sees search engines and BI coming together as the world of unstructured data is reeled in to the business need for intelligence. “At the moment 80-85 per cent of the data stored on computers is unstructured,” he said, “so it would be good to have a common framework that will allow users to analyse a record or historical data to identify problems, issues or trends – analysing structured and unstructured data together.”

The paradigms used to interface with that combined framework may look like a search engine but they will be different. The text related search technologies have been, until recently, solely built on keywords not semantics. There have been some sophisticated algorithms developed that can look at keywords in a context, recognising company names, technical components and the like. But they were developed in a proprietary fashion and that, Mattos suggests, is why they have never taken off.

Security for virtualized datacentres

More from The Register

next story
Microsoft to bake Skype into IE, without plugins
Redmond thinks the Object Real-Time Communications API for WebRTC is ready to roll
Microsoft promises Windows 10 will mean two-factor auth for all
Sneak peek at security features Redmond's baking into new OS
Mozilla: Spidermonkey ATE Apple's JavaScriptCore, THRASHED Google V8
Moz man claims the win on rivals' own benchmarks
FTDI yanks chip-bricking driver from Windows Update, vows to fight on
Next driver to battle fake chips with 'non-invasive' methods
PEAK APPLE: iOS 8 is least popular Cupertino mobile OS in all of HUMAN HISTORY
'Nerd release' finally staggers past 50 per cent adoption
DEATH by PowerPoint: Microsoft warns of 0-day attack hidden in slides
Might put out patch in update, might chuck it out sooner
Ubuntu 14.10 tries pulling a Steve Ballmer on cloudy offerings
Oi, Windows, centOS and openSUSE – behave, we're all friends here
Was ist das? Eine neue Suse Linux Enterprise? Ausgezeichnet!
Version 12 first major-number Suse release since 2009
prev story

Whitepapers

Why cloud backup?
Combining the latest advancements in disk-based backup with secure, integrated, cloud technologies offer organizations fast and assured recovery of their critical enterprise data.
A strategic approach to identity relationship management
ForgeRock commissioned Forrester to evaluate companies’ IAM practices and requirements when it comes to customer-facing scenarios versus employee-facing ones.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.
The hidden costs of self-signed SSL certificates
Exploring the true TCO for self-signed SSL certificates, including a side-by-side comparison of a self-signed architecture versus working with a third-party SSL vendor.