Red Hat juices speed freak MRG Linux
Shadowman gets the message
Red Hat updated its core Enterprise Linux operating system stack to 6.2 in December and its Enterprise Virtualization commercial-grade KVM server hypervisor to 3.0 last week. Now Shadowman has polished up a new release of a special stack of Linux and systems software called MRG aimed at hard-core messaging, real-time, and high performance computing workloads where generic Linux just don't cut it.
Enterprise Message, Real-time and Grid, as MRG is officially called, debuted back in December 2007 when a number of compute grid and financial trading customers convinced Red Hat that it needed a special version of its Enterprise Linux stack that had a real-time kernel that could provide consistent and predictable performance at very low latencies. Novell, which owned SUSE Linux at the time, had just put its SUSE Linux Enterprise Real Time, or SLERT, variant of its Linux stack into the field, so Red Hat had to jump in, too.
In addition to offering a real-time kernel, Red Hat's MRG weaved in the Advanced Message Queuing Protocol (AMQP), an open source and standard message queuing stack akin to the messaging server brokers such as IBM's WebSphere MQ, Microsoft's Message Queuing Middleware, Tibco's Rendezvous financial transaction messaging, and the Java Messaging Service. Eventually, the Condor open source compute grid project from the University of Wisconsin was added to the platform, giving it its G at the end.
In December 2009, Red Hat updated MRG to the Linux 5.4 code base, and has been tweaking it here and there over the past two years. For instance, MRG 2.0, launched last June, was based on RHEL 6.1, which debuted back in May 2011 and which already supports most of the current and impending x86 iron (such as the new Opteron 4200 and 6200 processors from AMD and the impending Xeon E5 processors from Intel) as well as other technologies such as PCI-Express 3.0 slots and USB 3.0 ports.
The big change with Enterprise MRG 2.1, as you can see in the release notes, is that the underlying RHEL stack is sporting the Linux 3.0 kernel. It is based on the "Santiago" RHEL 6.1 Linux variant Red Hat put out last December and hardened as the core of its RHEV 3.0 hypervisor.
As has happened with past MRG rollouts, the real-time Linux kernel and the updated Condor grid software is shipping now, but the AMQP messaging components for MRG 2.1 are not shipping for a few months yet.
The kinds of companies that use MRG – high performance computing clusters, high frequency trading systems, and other kinds of performance freaks – have performance anxiety and they need a lot of reassurance from measuring performance. So Red Hat's engineers have cooked up some less-intrusive ways of doing this using event-based sampling that also reduces some of the uncertainty in measuring using prior methods.
The main new features in MRG 2.1 are that the messaging broker can now maintain sequential order of messages across distributed receivers (that's so you can maintain proper transaction orders in message-based systems, which is important in figuring out who made and who lost money in a trading system.) The Condor grid portion of the MRG stack now has SSL encryption through its Aviary API, but it is not clear from the release notes (which are a bit sparse) what level of Condor is supported in the stack.
The grid scheduler can now be clustered for high availability using Red Hat's own cluster suite (losing a scheduler is a very bad thing), and the AMQP messaging stack is now integrated with Red Hat's own JBoss Enterprise SOA and Application platforms as well as Microsoft Visual Studio 2010 and supports IPv6 as a transport. There are a bunch of bug fixes for the messaging, real-time, and grid portions of the code, as you expect in any new release.
Red Hat has never been clear about what Enterprise MRG costs, and that has not changed with MRG 2.1. The idea way back in 2008 was for a MRG support contract to cost roughly twice as much as a RHEL support license of equivalent depth and speed. If a company is so big on openness and open source, you'd think it would publish a price list. I mean, even Oracle does that much. ®
The clue is in the Latency
If we look at the Financial Trading Systems where this sort of OS is used, a guaranteed latency is the name of the game.
These systems broadcast data to tens if not hundreds of clients. The AMQP message queueing software gives that sort of guaranteed low latency messaging these clients demand.
I find the comparison with other messaging systems rather funny.
Websphere MQ? Yes but only if you use the Low Latency variant.
MSMQ? Are you having a laugh?
JMS? Well, OOTB JMS is quite slow but you can use lighter weight transports so probably Yes.
Once you get down to guaranteed latencies in the low microsecond range, pretty well everything can be classed as an RTOS.
The trouble with using a general purpose OS (eg Linux, Unix or Windows) as an RTOS is that in many cases other non Real Time applications are loaded onto RT systems. When that happens, they don't become RT any more.
Then we get to RT on VM's (apart from ones that get very close to the Hardware such as z/OS & IBM pSeries LPARS) and in the Cloud. For that, all I can say is, are you having a laugh? However there is no doubt that in time that will happen.
I think this is the one used by the LSE in their new $30 million system, after they toast out the $50 million Windows system that crashed.
"Windows certainly is no "hard realtime" capable OS, but it is at least good enough to view movies properly. Something Linux seems to be challenged on occassion."
Excellent - so I was actually looking for technical feedback on something I am not well informed about, and you come back with some irrelevant drivel about watching movies. Genius.