Feeds

Structured data is boring and useless

And unstructructured data? Hmmm, sexy

Security and trust: The backbone of doing business over the internet

Editor's blog We all know that structured data is boring and useless; while unstructured data is sexy and chock full of value. Well, only up to a point, Lord Copper.

Genuinely unstructured data can be a real nuisance - imagine extracting the return address from an unstructured letter, without letterhead and any of the formatting usually applied to letters.

A letter may be thought of as unstructured data, but most business letters are, in fact, highly-structured. This issue exercises Duncan Pauly, founder and chief technology officer of Coppereye, which sells indexing technology that exploits data structures, even when the data is in an unstructured container.

Duncan comes from the Oracle database world and has a deep understanding of data analysis and structure - and he finds that a fundamental knowledge of database and data structures is becoming increasingly rare.

I have to agree. At an IDC conference on Business Performance Management and Business Intelligence yesterday, I heard a speaker use Google Earth as an example of the use of unstructured data: "You just type in your postcode..."

Hold on a second. The data is geocoded by postcode of all things and identified by latitude and longitude (and, presumably, there's catalogued metadata associated with the photographs) - how on Earth [sorry] can it be called unstructured?

And, no, Google has not rendered data analysis and so on obsolete - my wife, an information scientist, never tires of pointing out that an unstructured search engine should be the last resort of the researcher, not the first. If you understand the structure and semantics of your data, there are usually more effective ways of retrieving the data you need than such a search engine can provide.

So, Duncan, what is the issue that's concerning you, exactly?

The labels "structured data" and "unstructured data" are often used ambiguously by different interest groups; and often used lazily to cover multiple distinct aspects of the issue. In reality, there are at least three orthogonal aspects to structure:

  • The structure of the data itself.
  • The structure of the container that hosts the data.
  • The structure of the access method used to access the data.

These three dimensions are largely independent and one does not need to imply another. For example, it is absolutely feasible and reasonable to store unstructured data in a structured database container and access it by unstructured search mechanisms.

What does that actually mean for the user of the data?

Unstructured data I take to be a sequence of tokens that require human interpretation or heuristics to derive semantic content. Structured data, however, has associated or implied metadata to define or label its semantics. Typically, unstructured data tends to be textual, where structured data tends contain vectors of quantitative scalar or referential identities.

OK, so (as I take it) truly unstructured data resists automation, a human has to be involved to provide the semantics necessary. The processing of structured data, even in unstructured containers, can be automated routinely - which is the business of developers. Perhaps we should throw this subject open to the readers of this blog; who are, presumably, mostly developers - what aspects of this issue do you think they should consider?

Well, perhaps:

  1. Do storage vendors, database vendors and search vendors misuse the "structured/unstructured" labels, and why?
  2. As search-engine user interfaces become more widely used and data-stores federate databases proper and document-style datastores, do we need to think more clearly about the distinction between unstructured and structured data?
  3. What are the implications of separating data-models from container models from access models? An RDBMS with an XML engine and a BLOB datatype, for example, stores a lot more than just relational data.

All right, all you practising developers. It's over to you...

Providing a secure and efficient Helpdesk

More from The Register

next story
New 'Cosmos' browser surfs the net by TXT alone
No data plan? No WiFi? No worries ... except sluggish download speed
'Windows 9' LEAK: Microsoft's playing catchup with Linux
Multiple desktops and live tiles in restored Start button star in new vids
iOS 8 release: WebGL now runs everywhere. Hurrah for 3D graphics!
HTML 5's pretty neat ... when your browser supports it
'People have forgotten just how late the first iPhone arrived ...'
Plus: 'Google's IDEALISM is an injudicious justification for inappropriate biz practices'
Mathematica hits the Web
Wolfram embraces the cloud, promies private cloud cut of its number-cruncher
Mozilla shutters Labs, tells nobody it's been dead for five months
Staffer's blog reveals all as projects languish on GitHub
SUSE Linux owner Attachmate gobbled by Micro Focus for $2.3bn
Merger will lead to mainframe and COBOL powerhouse
iOS 8 Healthkit gets a bug SO Apple KILLS it. That's real healthcare!
Not fit for purpose on day of launch, says Cupertino
prev story

Whitepapers

Secure remote control for conventional and virtual desktops
Balancing user privacy and privileged access, in accordance with compliance frameworks and legislation. Evaluating any potential remote control choice.
WIN a very cool portable ZX Spectrum
Win a one-off portable Spectrum built by legendary hardware hacker Ben Heck
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
The next step in data security
With recent increased privacy concerns and computers becoming more powerful, the chance of hackers being able to crack smaller-sized RSA keys increases.