Original URL: http://www.theregister.co.uk/2010/03/19/programming_open_data/

Microsoft, 'open' data, and the curse of open source

Thanks a lot, HTML5

By Gavin Clarke

Posted in Software, 19th March 2010 17:02 GMT

Do the rise of cloud computing and the outbreak of peace on open standards in the browser mean programmers will be forced to find new ways to make money online?

In the last few days, Microsoft surrendered to common sense by announcing that Internet Explorer will finally embrace common standards with HTML5.

It was a critical moment that potentially means the end of the lock-in that enabled Microsoft and others to charge for their coding work. While IE might be free, billions of hours were spent on custom coding as Microsoft, partners, and an entire industry built web sites, applications and online services first for Microsoft's browser and then for everybody else that bothered to adhere to web standards.

The disappearance of this walled garden - due to happen with IE 9 - has coincided with the feverish rise of smart-phone makers and service providers falling over themselves to give consumers access to things like Twitter and Facebook on their handsets.

Apple, Palm, Microsoft, Google, Blackberry, and Symbian are fighting to prove they too offer access to exactly the same handful of social networks so their users can also Tweet their way through important business meetings.

Increasingly, this battle to carry the same services requires both integration between services and between those services and the operating system.

Palm, for example, has Synergy in its webOS that links and merges contacts, calendar information and messages to avoid fumbling through different screens. On the other end of the scale, Microsoft's Windows Phone 7 Series has a long way to go on basic integration and multitasking, but we should expect it to catch up over time in classic Microsoft fashion.

This trend for integration is starting to reach into the world of the desktop. Lucid Lynx - the next version of the Linux desktop due imminently - will integrate Twitter and Facebook into the software, according to Ubuntu founder Mark Shuttleworth.

That should merge the desktop and online worlds so people don't have to fire up their browser or a separate application to use their social applications.

All this means that while each mobile and desktop operating system is unique and highly complex - facts that let developers charge for the time and work that goes into building their software - they are all targeting exactly the same data by writing to exactly the same APIs, be they Facebook's or Google's vast repository of search queries or geo-location information.

This trend of writing to what speakers at the Open Source Business Conference (OSBC) this week in San Francisco, California called "open data" - never mind the whole issue of proprietary cloud lock in - is being further driven by giants like Microsoft and Google.

In health, for example, you have Microsoft HealthVault and Google Health - two growing repositories that are seeing Microsoft and Google set themselves up as massive gatekeepers of information. They have recognized that in the information age, survival comes not by adding more features to applications or operating systems but by owning the information itself and then letting others access it. Talk about buying your way into the future.

Oh my, OData

Microsoft went further this week. It announced OData, a set of plumbing APIs to help applications consume such information. It's the company's extension of Atom and an alternative to Google's Rest-based GData. Also announced were 25 data sets and SDKs as part of Microsoft's Project Dallas that it's willing to let developers use in their applications.

Is coding as we know it reaching some kind of commoditization end point, where the rush to connect to exactly the same Facebook or Google data means looking for new to ways to make money?

Yes, according to one Adobe executive speaking at an OSBC panel on the web platform. Adobe director of open-source standards Dave McAllister, said: "How the hell do we make money on open data - that's the thing, that's really the question that comes in to play. We can link all these things together ... [but] you have to start moving from the model of money is made in the program or in the presentation to the money is made in the data."

According to McAllister, the money comes in finding ways to build services that capitalize on all the raw data that's rapidly accumulating inside the massive server farms of Facebook, Google or Microsoft and has been enriched by people helpfully and unselfconsciously adding tags, geolocation information or simply emailing friends or colleagues.

"There's money to me made out of the billions of data points, there's also money to be made out of the personal service points," McAllister said. He confessed he had no answers as to how to do this.

Palm director of developer relations Dion Almaer told OSBC he believes open-standards like HTML 5 provide a "unifying platform" capable of delivering the same experience based on all this information across a range of different devices, regardless of their underling architectures.

Let's not get carried away here. There will always be a need for coding and this world of open information will not become the default programming paradigm.

In classic Silicon Valley fashion, the OSBC debate focused on web- and cloud-based services, a world defined by massive volumes of data, storage and processing in huge data centers from tier-one providers. This is not surprising given that Google's on the doorstep and venture capitalists down the road at Sand Hill set the pace by telling start-ups to get in the cloud if they want funding. A few years back the VCs also told Valley hopefuls to get with Software as a Service (SaaS), open source and the dot-com wave.

For most ordinary developers or users in the real world, reality is not cloud computing - it's their desktop or server. In this world, companies will choose on-premises software instead of cloud computing for a variety of reasons ranging from corporate inertia, lack of budgets, and internal politics to paranoia over security.

Brian Goldfarb, Microsoft's lead product manager for web platform tools believes - correctly - that this means there will continue to be a role for both Java and Microsoft's .Net, along with the current crop of highly agile frameworks, because they both serve different customers. Microsoft's challenge is releasing code fast enough to satisfy agilistas [wha? -Ed] while being slow enough for the out-of-shape enterprise using .NET to keep up.

There's also plenty of scope for the industry to screw things up as it nearly did in the early 2000s. We have nimble, modular, and light-weight frameworks like PHP and Rest in spite of - not because of - companies like Microsoft and IBM who cooked up a heavy set of WS-web services specs with the goal of outflanking the competition at the time (mostly Sun Microsystems).

The standards bodies themselves responsible for generating things like HTML 5, meanwhile, are notorious for bulking out specs as members shoehorn what they want into the main specification.

Almaer said he hoped industry giants have learned their lesson from adding more and more features to technologies that had created what he called the "WS-Death Star oriented crap" of the early 2000s. McAllister was less optimistic: "It will get big. Hopefully we leaned our lesson, but it will get big," he said of cloud technologies behind open information.

There's also a threat from Apple. Almaer said while HTML 5 provides a unifying force, the iPad - if it takes off like the iPhone - could throw up another proprietary hardware and software island that means a return to the past of lock-in on the web.

Beware the giant sucking sound

Government, too, could hamper things. One concern is the local laws governments begin applying to companies like Microsoft and Google as they hoover up more and more data. Government could finally wake up to value of all that information and recognize the sleeping giant of the problem of whether it's desirable for a handful of super service providers to be the gatekeepers of their citizens' information and a new force in society. Google and Facebook have made a number of policy and statement gaffes on privacy in relation to their services that have finally caused the mask to slip.

"Internationally we could run into boundaries that are setup by countries and governments that could cause this to fragment in a political spectrum rather than a technology spectrum. That scares me, because most engineers are logical and sensible - politicians? I'm not so sure," McAllister said.

Microsoft's decision to surrender on HTML5 and tear down its browser walled garden was a big move. It comes as "open" technologies such as HTML, Rest, and PHP are on the march - a move that's forced companies with their own proprietary framework to become honest.

It's a world away from computing of the 1980s and 1990s where vendors brazenly talked of lock-in as a way to charge customers and "own" the developers that added value to their platforms.

As the walls come down - and vendors give away their advantage in the rush to connect services - the challenge for those who aren't Facebook, Google, or Microsoft and who aren't the gatekeepers of all that information will be how to turn access to the same data as everybody else gets into gold.

It's a challenge that has kept people working late evenings and long weekends in that other great bastion of "open" - open source code. ®