Feeds

All aboard the Enterprise Service Bus

Event processing, anyone?

3 Big data security analytics techniques

At SOA ML4 and 5 the key technologies are, respectively, Business Activity Modelling or Event Stream Processing (ESP); and Event Stream Automation for optimisation. This is interesting, cutting-edge stuff (best exemplified by event-driven securities trading) – and probably provides Sonic with “more useful services” differentiation from its competition.

In essence, rather than analysing static records of the day's events, ESP looks at the pattern of business events in (very) near-real-time and then fires off alerts, or even instigates actions, quickly enough to allow emerging situations to be controlled before they become critical. At a recent Gartner Integration conference (Barcelona 2005), both SeeBeyond and WebMethods identified event processing of some sort as the next driver for integration and Progress' Real Time Division is well-placed to exploit this trend.

It has the SOA concept to hold everything together, the Sonic ESB to pass events along to where they can be acted on and object-oriented database technology (originally from ObjectStore) to implement an Event Store, capable of storing multitudes of real-time events and making them accessible in real-time too. The event store is fundamental to ESP and Progress's Real Time Division implements it as an in-memory database mapped onto disk storage for persistence using virtual-memory-like techniques; with indexes for after-the-fact, static, analysis.

Progress acquired its fundamental ESP knowledge by buying Apama, a leading innovator in algorithmic equities and futures trading, which is now extending its capabilities into the FX (foreign exchange) markets.

According to Mark Palmer, VP, Progress Real Time Division ESP, ESP as adds a missing part to an event-driven SOA - events are intrinsic to the ESB concept.

ESP can be differentiated from previous technologies, such as Business Activity Monitoring dashboards (although ESP drives dashboards too), by its event processing language and an engine that can output complex or derived events, he says. Complex events are based on processing real time events against, possibly, data in a conventional database and/or previous events in the Event Store and then generating new events, that can be transmitted to other applications over, for example, the Sonic ESB.

So, what can you use this new technology for? Well, clever financial trading and compliance checking are the current hot applications. A trader may split up very large stock purchases or sales so as to avoid moving the market price; but an ESP engine might notice a series of fixed size purchases a fixed distance apart, infer the placing of a large deal in total and react accordingly.

Split times

So, traders will start randomising the splitting up of such deals and the ESP will have to get cleverer. At the same time, ESP can be used to react to events such as, say, two brokers selling stock at exactly the same time (if deliberate, this can be used to manipulate prices; but it might happen by chance) and to enforce dealing rules generally.

Palmer, however points out that there are many other applications – processing the stream of events from RFID (Radio Frequency ID) tag readers for example, possibly acting on the discovery that a lot of tagged objects coming into a distribution facility and rather fewer are leaving each stage of breakdown and packaging. For example, Delta Airlines' “Delta Nervous System", which analyses weather patterns, aircraft positions etc, determines necessary flight re-routings and generates new events such as passenger rebookings, all in the interests of providing a better passenger experience (and more loyal customers), is another example of what you can do with event processing.

Nevertheless, ESP will have its issues. How easily can you test and validate complex event processing logic, for instance – is it possible that it works on a few events during testing but that aberrant behaviours emerge when hundreds of thousands of events, and derived events, are being issued and acted on in real time?

And, of course, there is governance - how do you show regulators that your real-time processing always followed the rules (which is why the event store, and an ability to replay events afterwards, is important)? We may suspect that mature SOA services such as ESP will require a certain level of maturity from the organisations operating them – although this is probably true of any advanced technology. ®

SANS - Survey on application security programs

More from The Register

next story
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Oh no, Joe: WinPhone users already griping over 8.1 mega-update
Hang on. Which bit of Developer Preview don't you understand?
Microsoft lobs pre-release Windows Phone 8.1 at devs who dare
App makers can load it before anyone else, but if they do they're stuck with it
Half of Twitter's 'active users' are SILENT STALKERS
Nearly 50% have NEVER tweeted a word
Internet-of-stuff startup dumps NoSQL for ... SQL?
NoSQL taste great at first but lacks proper nutrients, says startup cloud whiz
Ditch the sync, paddle in the Streem: Upstart offers syncless sharing
Upload, delete and carry on sharing afterwards?
New Facebook phone app allows you to stalk your mates
Nearby Friends feature goes live in a few weeks
Microsoft TIER SMEAR changes app prices whether devs ask or not
Some go up, some go down, Redmond goes silent
Batten down the hatches, Ubuntu 14.04 LTS due in TWO DAYS
Admins dab straining server brows in advance of Trusty Tahr's long-term support landing
prev story

Whitepapers

SANS - Survey on application security programs
In this whitepaper learn about the state of application security programs and practices of 488 surveyed respondents, and discover how mature and effective these programs are.
Combat fraud and increase customer satisfaction
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Top three mobile application threats
Learn about three of the top mobile application security threats facing businesses today and recommendations on how to mitigate the risk.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.