IBM rejigs Platform control freakery for supers

Hadoop MapReduce clone, new HPC stacks ship this month

Internet Security Threat Report 2014

Dances with elephants

Speaking of Symphony, Platform was just getting ready to shim a Hadoop-compatible MapReduce stack on top of Symphony, which it said was far better and more efficient than the open source Hadoop and its Hadoop Distributed File System, when Big Blue bought the company.

The batch-oriented LSF was built for scale, not for speed and certainly not for running Java messaging applications that are the backbone of financial services organizations, doing Monte Carlo simulations, risk analytics, and pricing algorithms for financial instruments. That's what Symphony was created for.

Back in May 2011, the then-independent Platform had embiggened the grid size with the Symphony 5.1 release and had previewed a variant of Symphony called Platform Workload Manager for MapReduce, which supported all of the MapReduce APIs in the Hadoop stack, but ran the actual Mapping and Reducing on top of Symphony, which is wickedly faster than the real Hadoop, according to Platform.

With Symphony 5.2, IBM is rolling this MapReduce support into the core product instead of making it a standalone product as Platform was going to do, Muelhoefer tells El Reg. Symphony is also being broken into a bunch of different editions, thus:

Editions of IBM's Symphony Java grid software

Editions of IBM's Symphony Java grid software (click to enlarge)

The MapReduce functions are only going to be available in the Developer and Advanced Editions. The Advanced Edition also includes the data affinity function that Platform had previously added to the Synphony stack, which moved computational work to where the data is rather than moving the data to where the free compute cycles are in a cluster. (MapReduce has a similar protocol, by the way.) This is a dot release of the Symphony code, and the software still scales to 10,000 cores per application and 40,000 cores per cluster, as the Symphony 5.1 release did.

Symphony schedules jobs to run on cores in milliseconds, compared to the seconds or minutes it takes with LSF, and can handle as many as 400 million tasks per day. The kind of performance that NASDAQ could have used on the day Facebook went public. (Hopefully, NASDAQ is not a Symphony reference account.)

The rejiggered LSF V8.3, Platform MPI V8.3, and Symphony 5.2 software will all be available on June 15. Pricing information was not available at press time.

Stack it up and cluster it

In a related announcement, IBM's supercomputer whizzes have also updated the Intelligent Cluster pre-fabbed HPC cluster stacks with a slew of new hardware, including the rejigged Platform Computing software and new servers, storage, and switches from IBM and its partners.

The stacks, known as the Intelligent Cluster 12B release, are not hard bundles so much as a la carte menus that customers can pick from and have IBM use to build a prefabbed cluster and drop it onto your loading dock. The latest bundles include IBM's own iDataPlex dx360 M4 servers, which have been refreshed with Intel's latest Xeon E5-2600 processors and Nvidia's latest Tesla K10 GPU coprocessors. The clusters can also be made from the System x3750 (four-socket Xeon E5-2600) or x3630 (two-socket Xeon E5-2400) servers, or the BladeCenter HS23 blade server (two-socket Xeon E5-2600).

IBM is bundling in its own RackSwitch G8316 (40GE), G8264 (10GE), and G8124E (Gigabit) switches and its forthcoming Platform HPC 3.2 for System x Express Edition and Platform Cluster Manager Advanced Edition to the stack, as well as its Global Parallel File System 3.5A. Ethernet switches from Brocade Communications and InfiniBand switches and adapters from Mellanox Technologies are options in the 12B HPC stacks, as is DataDomain Networks' SFA12K InfiniBand storage controller.

The Intelligent Cluster 12B stacks also can be the heart of finished stacks that IBM's factory can slap together to run pre-integrated application software such as IBM's own SmartCloud stack or SAP's Business Warehouse or HANA in-memory appliance.

There are also solution reference architectures (cookbooks instead of ready to eat) for the 12B iron, including IBM's own BigInsights variant of Hadoop, Schlumberger Eclipse (for petroleum exploration) mpiBLAST (for life sciences), ANSYS Fluent (for auto and aerospace manufacturing), Autodesk Vault (CAD/CAE), and OpenStack private and public clouds.

The Intelligent Cluster 12B stacks will be ready to roll on June 20. ®

Beginner's guide to SSL certificates

More from The Register

next story
Docker's app containers are coming to Windows Server, says Microsoft
MS chases app deployment speeds already enjoyed by Linux devs
'Hmm, why CAN'T I run a water pipe through that rack of media servers?'
Leaving Las Vegas for Armenia kludging and Dubai dune bashing
'Urika': Cray unveils new 1,500-core big data crunching monster
6TB of DRAM, 38TB of SSD flash and 120TB of disk storage
Facebook slurps 'paste sites' for STOLEN passwords, sprinkles on hash and salt
Zuck's ad empire DOESN'T see details in plain text. Phew!
SDI wars: WTF is software defined infrastructure?
This time we play for ALL the marbles
Windows 10: Forget Cloudobile, put Security and Privacy First
But - dammit - It would be insane to say 'don't collect, because NSA'
Oracle hires former SAP exec for cloudy push
'We know Larry said cloud was gibberish, and insane, and idiotic, but...'
Symantec backs out of Backup Exec: Plans to can appliance in Jan
Will still provide support to existing customers
prev story


Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Why cloud backup?
Combining the latest advancements in disk-based backup with secure, integrated, cloud technologies offer organizations fast and assured recovery of their critical enterprise data.
Win a year’s supply of chocolate
There is no techie angle to this competition so we're not going to pretend there is, but everyone loves chocolate so who cares.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Intelligent flash storage arrays
Tegile Intelligent Storage Arrays with IntelliFlash helps IT boost storage utilization and effciency while delivering unmatched storage savings and performance.