Feeds

Sorting the ETL men from the boys

Diverging paths

Internet Security Threat Report 2014

Comment The ETL (extract, transform and load) market, far from commoditising, is diverging. To begin with, ETL is no longer an appropriate term to use, both because operations are no longer limited to the order indicated but also because the technology encompasses far more than just moving data into a warehouse. However, I don't like the alternatives such as "data movement" and "data transfer" much, while "data integration" is too broad, so I guess we are stuck with ETL. However, this is by no means the only area of divergence.

Perhaps the most obvious change in the market is the growth in code generating products and there is now a clear split in the market between black box solutions and code generating approaches. While the former saw off the previous generation of code-based products a decade ago, it is by no means clear cut that they will do so again: SQL and Java are much more portable than the Cobol-based products of the early nineties.

Code-based approaches are also helped by the many ISVs that want the ability to embed specific ETL capabilities within their own products, and there are a number of newer ETL suppliers specifically targeting this market either directly or in a complementary fashion. For example, Baycastle focuses on doing things like moving data into contact management systems.

Another major change has been the advent of Open Source (Clover and Kinetic Networks' KETL) products and even shareware products (DB Software), which should help to drive user acceptance of the "don't hand code" message and which can only benefit everybody.

However, returning to the established players versus the new entrants discussion, the big advantage that the former have is that they provide lots of complementary functionality, notably with data quality, enterprise information and application integration and so on, though this is not limited to black-box solutions (witness Sunopsis).

Finally, the latest area of divergence is in the ability to support the extraction, transformation and loading of unstructured and semi-structured content. Of course, the concept of unstructured content is a nonsense – if it was really unstructured it would collapse into a heap – but, for the purposes of this discussion I mean Word and pdf documents and the like on the one hand (unstructured) and HIPAA, EDIFACT, SWIFT and similar documents (semi-structured on the other).

Of course, this is not entirely new: Ascential has had abilities in the area of semi-structured data ever since it bought Mercator (now DataStage TX), while Hummingbird has offered the ability to extract unstructured content for some time, largely because it is the only ETL vendor that is also a major content/document management provider. However, Informatica has now added this capability as generic functionality and other vendors are likely to follow suit.

If the ability to build applications that combine content and data is to be the major growth area that many suspect that it will be, then the ability to support ETL functions against content as opposed to data is likely to be a defining factor and will sort out the ETL men from the boys.

Copyright © 2005, IT-Analysis.com

Choosing a cloud hosting partner with confidence

More from The Register

next story
Netscape Navigator - the browser that started it all - turns 20
It was 20 years ago today, Marc Andreeesen taught the band to play
Sway: Microsoft's new Office app doesn't have an Undo function
Content aggregation, meet the workplace ... oh
Do Moan! MONSTER 6-day EMAIL OUTAGE hits Domain Monster
Customers freaked out by frightful service
Sign off my IT project or I’ll PHONE your MUM
Honestly, it’s a piece of piss
Return of the Jedi – Apache reclaims web server crown
.london, .hamburg and .公司 - that's .com in Chinese - storm the web server charts
NetWare sales revive in China thanks to that man Snowden
If it ain't Microsoft, it's in fashion behind the Great Firewall
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Why cloud backup?
Combining the latest advancements in disk-based backup with secure, integrated, cloud technologies offer organizations fast and assured recovery of their critical enterprise data.
Win a year’s supply of chocolate
There is no techie angle to this competition so we're not going to pretend there is, but everyone loves chocolate so who cares.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Intelligent flash storage arrays
Tegile Intelligent Storage Arrays with IntelliFlash helps IT boost storage utilization and effciency while delivering unmatched storage savings and performance.