IBM rejigs Platform control freakery for supers

Dances with elephants

Speaking of Symphony, Platform was just getting ready to shim a Hadoop-compatible MapReduce stack on top of Symphony, which it said was far better and more efficient than the open source Hadoop and its Hadoop Distributed File System, when Big Blue bought the company.

The batch-oriented LSF was built for scale, not for speed and certainly not for running Java messaging applications that are the backbone of financial services organizations, doing Monte Carlo simulations, risk analytics, and pricing algorithms for financial instruments. That's what Symphony was created for.

Back in May 2011, the then-independent Platform had embiggened the grid size with the Symphony 5.1 release and had previewed a variant of Symphony called Platform Workload Manager for MapReduce, which supported all of the MapReduce APIs in the Hadoop stack, but ran the actual Mapping and Reducing on top of Symphony, which is wickedly faster than the real Hadoop, according to Platform.

With Symphony 5.2, IBM is rolling this MapReduce support into the core product instead of making it a standalone product as Platform was going to do, Muelhoefer tells El Reg. Symphony is also being broken into a bunch of different editions, thus:

The MapReduce functions are only going to be available in the Developer and Advanced Editions. The Advanced Edition also includes the data affinity function that Platform had previously added to the Synphony stack, which moved computational work to where the data is rather than moving the data to where the free compute cycles are in a cluster. (MapReduce has a similar protocol, by the way.) This is a dot release of the Symphony code, and the software still scales to 10,000 cores per application and 40,000 cores per cluster, as the Symphony 5.1 release did.

Symphony schedules jobs to run on cores in milliseconds, compared to the seconds or minutes it takes with LSF, and can handle as many as 400 million tasks per day. The kind of performance that NASDAQ could have used on the day Facebook went public. (Hopefully, NASDAQ is not a Symphony reference account.)

The rejiggered LSF V8.3, Platform MPI V8.3, and Symphony 5.2 software will all be available on June 15. Pricing information was not available at press time.

Stack it up and cluster it

In a related announcement, IBM's supercomputer whizzes have also updated the Intelligent Cluster pre-fabbed HPC cluster stacks with a slew of new hardware, including the rejigged Platform Computing software and new servers, storage, and switches from IBM and its partners.

The stacks, known as the Intelligent Cluster 12B release, are not hard bundles so much as a la carte menus that customers can pick from and have IBM use to build a prefabbed cluster and drop it onto your loading dock. The latest bundles include IBM's own iDataPlex dx360 M4 servers, which have been refreshed with Intel's latest Xeon E5-2600 processors and Nvidia's latest Tesla K10 GPU coprocessors. The clusters can also be made from the System x3750 (four-socket Xeon E5-2600) or x3630 (two-socket Xeon E5-2400) servers, or the BladeCenter HS23 blade server (two-socket Xeon E5-2600).

IBM is bundling in its own RackSwitch G8316 (40GE), G8264 (10GE), and G8124E (Gigabit) switches and its forthcoming Platform HPC 3.2 for System x Express Edition and Platform Cluster Manager Advanced Edition to the stack, as well as its Global Parallel File System 3.5A. Ethernet switches from Brocade Communications and InfiniBand switches and adapters from Mellanox Technologies are options in the 12B HPC stacks, as is DataDomain Networks' SFA12K InfiniBand storage controller.

The Intelligent Cluster 12B stacks also can be the heart of finished stacks that IBM's factory can slap together to run pre-integrated application software such as IBM's own SmartCloud stack or SAP's Business Warehouse or HANA in-memory appliance.

There are also solution reference architectures (cookbooks instead of ready to eat) for the 12B iron, including IBM's own BigInsights variant of Hadoop, Schlumberger Eclipse (for petroleum exploration) mpiBLAST (for life sciences), ANSYS Fluent (for auto and aerospace manufacturing), Autodesk Vault (CAD/CAE), and OpenStack private and public clouds.

The Intelligent Cluster 12B stacks will be ready to roll on June 20. ®