Data Centre


IBM man goes deep on why they're all shiny OpenCAPI people

More details emerge at HPC powwow in Switzerland

By Dan Olds, OrionX


HPC Blog IBM gave the audience a deeper dive into the OpenCAPI initiative and hardware at the recently concluded HPC Advisory Council annual meeting in Lugano, Switzerland.

OpenCAPI is a new connection type that gives a high bandwidth, low latency, connection for memory, accelerators, network, storage, and other devices like ASICs.

One of the design goals for CAPI (Coherent Accelerator Processor Interface) was to minimise latency – shooting for "equivalent effective latency of DDR standards". They've done a good job of reaching that goal, with very low 10 nanosecond latency for devices like accelerators and even achieving 5ns latency for advanced memory.

The latency numbers are a pretty big deal – POWER8 round trip latency is about 10ns, and that's very sporty for a processor. Even more important is that the 10ns latency of OpenCAPI beats PCIe like a drum, since a typical PCIe round trip incurs a 100ns latency penalty.

When it comes to bandwidth, OpenCAPI today offers 25GBps while PCIe gen 3 currently offers 32GBps. The speaker, IBM's Jeff Stuecheli, explained that succeeding instances of OpenCAPI will run at 32GBps then 56GBps and higher.

He also pointed out that OpenCAPI is hardware agnostic, meaning it doesn't have any design lockouts, like, for example, some FPGAs, which only connect into the QPI channel of the motherboard through the processor socket. This limits customer choice in FPGAs and thus increases cost. Plus there's a potential reliability concern, since a failure in the FPGA would bring down the entire system.

Both a video of his talk and his slides are available here. In the rest of his discussion, he talks about how OpenCAPI devices operate in the virtual address spaces of the applications they support and how the virtual-to-physical address translation occurs in the CPU. He also goes into details about the protocol stack and the programming model.

This slide deck and discussion is much more detailed than other presentations I've seen about CAPI and OpenCAPI. It gives us more context and provides a fuller explanation of why other companies (Dell, HPE, Micron, AMD, plus others) have joined the OpenCAPI consortium. One notable absentee is Intel, which is understandable, since CAPI is a competitor to Intel's OmniPath product set and the venerable PCIe standard.

Sign up to our NewsletterGet IT in your inbox daily


More from The Register

Ah, breathe that fresh alpine air. And look over there, a majestic HPC Advisory Council

HPC Blog Beastmode crew hosts three-day conference in Lugano, Switzerland

Want to know more about HPC apps? This explicit vid has some answers

HPC Blog Page through this profiler...

Cluster-wrestling kids pimp their HPC rides in Frankfurt

HPC Blog This one goes to 11: More rig pr0nz from students at ISC17

HPC Advisory Council unveils Cloud HPC initiative

Webcast Ready for prime time?

Points tables unveiled: Who needs sport? Follow HPC kids' summer cluster cup final

HPC Blog Third trophy for South Africa

To heck with the laws of physics... we will squeeze more juice from these processors

HPC Blog An HPC boffin predicts system trends

Parallel programming masterclass with compsci maven online

HPC Blog Dr Panda's recent Swiss presentation free to view

Your chance to become a supercomputer superuser – for free

HPC Blog Tutorials, best practices at Lugano HPC-fest

South African cluster kids represent ENTIRE continent at HPC battle

HPC blog No pressure then. Plus, second time around for Costa Rica

Experienced HPC compo kids welcome eager Brazilians to cluster-wrestling match

HPC blog Foreign interlopers aim for China Cluster Crown