Compensating for SOA errors
I did it iWay
John Schlesinger is director of integration Solutions for iWay Software (part of Information Builders Inc) and has a long history in our business.
He’s been working on developing middleware since 1985, both at IBM (he worked on the CICS development team) and at iWay Software, where he’s now responsible for the iWay Business Services Provider. One of the good things about my current job is the opportunity to talk to such people (I wish I’d had such access when I was actually working in IT).
We were talking about “transaction processing” and about whether you can provide something similar in a loosely-coupled service-oriented environment. And, why you don't use 2-phase commit in SOA (basically, it's too slow and clumsy).
So, I was interested to see more-or-less the same points coming up in Mark Whitehorn's piece on Jim Gray - great architects obviously think alike.
The point is that transactions are good in an environment that you have full control over, but even there they have limits. When I was an IMS DBA, we were proud of recovering to the point of failure (and not losing data or referential integrity in the process - one of the benefits of transaction processing; and journaling, of course).
This was certainly preferable to recovering to a week-old database image copy and saying sorry to everybody who now had to re-enter their updates (something that some eCommerce sites seem all too happy with). However, if you send a cheque out of the bank, you can't recall it if something fails - although you can roll back the database updates involved to a consistent synch point. In the general case, if something goes wrong, as John and Jim both say, you sometimes have to run compensating logic. This means that you need to process forward, to correct the effects on the business of whatever went wrong - and get a person involved to sort out any remaining problems and apologise to the people you might inconvenience. So, if you have a ginormous eCommerce system, it had better not go wrong too often, as compensation won't always be trivial.
John expresses some nervousness about the current craze for composite applications built using business process execution languages of various flavours. He suggests checking out the slick demos vendors show you – and asking whether the vendor can demonstrate its error or retry processing and compensation logic. Now I'm nervous too.
But John also points out that the people he meets say that they aren't ever going to use these composite business process applications for critical line of business stuff, but only for admin apps - so now I'm less nervous. Only, I've seen lots of business leaders adopt trendy solutions that don't really work - and then the IT people have to sort it all out.
And you can make such composite applications work - John quotes Merrill Lynch as an example. Its composite applications, however, are designed to work together and compensate for errors correctly - the composite comprises only specific applications designed to fit together explicitly, which is fine, but not exactly the free-and-easy model that service orientation is supposed to enable.
John's approach to integration - which is what most business systems spend most of their time doing - is to push event-driven architectures rather than low-level request/reply solutions; and to firmly separate the application logic from the business integration logic. So, as I see it, you might have conventional application transactions to update the underlying databases (maintaining ACID properties) but you don’t publish them as Web Services. Instead, you publish a business-level service, which uses business level information - and submits any required application transactions - systems of record in the enterprise still often run on big mainframe databases - in the background. If any parts of this fail at the business level, then this event can be detected; and should be corrected at the business level.
iWay does have an ESB and a BPEL modelling tool these days, and I was quite impressed when I saw them at Summit (I was even more impressed by customers waiting for the iWay approach to ESB etc. to appear, because they trusted iWay not to let them down as a partner).
Nevertheless, it's good to hear a vendor recognising that its latest technology doesn’t entirely replace what we have already – and that it has to be used with a certain amount of architectural forethought ®.