Original URL: http://www.theregister.co.uk/2008/02/11/mashup_complexity/

Betamax 2.0: the future of mashups?

Complexity gets complicated

By John Hunt

Posted in Applications, 11th February 2008 11:02 GMT

Simplicity in software is, I believe, more than just a noble aim; it is essential for successful software projects. However, simplicity should not be assumed just because one particular technology or methodology is being used.

Mashups, discussed recently by Reg Dev reader Aubry Thonon, are a case in point. One element of the hype around mashups is they are simple to build because all you have to do is link together a few APIs and then you are done.

Life is never that simple, though. Building successful, effective, reliable and long- running mashup applications is not a trivial task, and - indeed - is something that creates its own architectural, organizational and implementation problems.

The current, popular, definition of a mashup is of a web-based application that combines data from two or more sources, into a single integrated solution. The most widely cited example of this is the combination of geospatial data from, say, Google Maps, with information on local businesses taken from an online directory or another data source. The end result allows the user to search for, say, a local baker and be presented with a map showing their locations and a brief summary of the sort of products they supply.

A mashup, then, is merely a new kind of integrated solution, albeit one using the web that can be built by anyone with the knowledge and access to a browser! Simple, huh?

No. Mashup developers will encounter integration issues well known to the software and database worlds, for which there are still no off-the-shelf solutions.

Infact, things are going to be a little more complicated in the mashup world. For example, unlike traditional integration, the suppliers of the source data that's mashed up are often not involved in the project, and may never have designed their data to be used in that way. This will create problems as systems do not automatically collaborate with each other.

Here, then, is my list of some of the most fundamental issues:

Lack of a single standard or technology: ok, some might call this choice, but developers are working with Asynchronous JavaScript and XML (AJAX) on the client, server side Java, XML-based web services, Representational State Transfer (REST) and RSS/ATOM data feeds. In addition, screen-scraping techniques are used to extract information from web sites that do not provide a more easily accessible data source. AJAX alone has 210 different frameworks, each with their own technical strengths and weaknesses, never mind differing levels of community support that will determine which survive for the long term and are really worth backing.

Complexity of architecture: combining multiple technologies, development styles and integration points in a single application does not a simple solution make. Indeed, while it is certainly possible to achieve a functioning system, the end result may well resemble a spaghetti of code using a pallet of interfaces and frameworks, rather than a cleanly engineered solution that's simple to use or to maintain.

Data integration: one fundamental assumption in any mashup is data will be integrated from two or more applications. However, what one system may refer to as a supplier another may call a vendor. What one system considers a number another may assume is a string. Thus integrating data from diverse sources is a very difficult job. For example:

Integration with browser functionality: if a mashup uses a browser then, to reach the broadest number of users, it must take into account the variations between different web browsers. And, as we all know, despite a general level of agreement on standards, some browsers are more compliant than others.

Don't get me wrong. Mashups are exciting and dynamic new systems that, since the early days of Google Maps, have rightly generated a lot of excitement.

However, they are not inherently simple or trivial to develop. As such, commercial organizations have sprung up to generously help solve these integration issues. Suddenly, it's looking like the old software consulting and integration business all over again, only using a greater modicum of openness and respect for standards.

Mashup developers, and those proposing mashups, need to be careful what they promise. They also need to give more thought to some traditional integration issues, such as data cleansing, and newer issues like persistence of the source feed.

If exponents of mashups over promise and under deliver, then users of the web will become disillusioned by mashups and avoid using them and building them. Mashups will then become the Betamax of the web generation: a great idea that lost a mass market.®