Why OpenCAPI is a declaration of interconnect fabric war
Any standard but Intel in another CPU-memory interconnect consortium
+Comment An OpenCAPI consortium has sprung into life, promising a new, open specification that can increase data center server performance up to 10x through the use of a new CPU-memory-IO adapter interconnect scheme – and it doesn't include Intel in its membership.
The Open Coherent Accelerator Processor Interface represents, the consortium says, a data-centric approach, and "provides an open, high-speed pathway for different types of technology – advanced memory, accelerators, networking and storage – to more tightly integrate their functions within servers."
It "puts the compute power closer to the data, removes inefficiencies in traditional system architectures to help eliminate system bottlenecks and can significantly improve server performance," according to the consortium.
The design specification is "built to minimize the complexity of high-performance accelerator design. Capable of 25Gbits per second data rate, OpenCAPI outperforms the current PCIe specification, which offers a maximum data transfer rate of 16Gbits per second."
The consortium's announcement release declares: "Many technology companies have developed innovative solutions that today's data center technology cannot effectively leverage due to limited legacy interfaces. New technologies such as storage class memory and accelerators to support emerging workloads do not fit well on existing interfaces and a closed, proprietary approach does not allow for full industry participation nor innovation."
OpenCAPI was founded by AMD, Google, IBM, Mellanox Technologies, and Micron. Other members include Dell EMC, HPE, Nvidia and Xilinx.
Several members have product/tech introductions coming:
- IBM plans to introduce Power9-based servers that leverage the OpenCAPI specification in the second half of 2017.
- IBM will enable members of OpenPOWER Foundation to introduce OpenCAPI-enabled products in the second half 2017.
- Google and Rackspace's new Zaius server under development, announced at the OpenPOWER Summit in San Jose, will leverage Power9 processor technology and plans to provide the OpenCAPI interface in its design.
- Mellanox plans to enable the new specification capabilities in its future products.
- Xilinx plans to support OpenCAPI-enabled FPGAs.
So it's mostly a Power9-driven effort currently.
The OpenCAPI consortium plans to make the OpenCAPI specification fully available to the public at no charge before the end of the year. Find out about this and membership details at opencapi.org.
Oh, and the supplier of the vast majority of server CPUs, Intel, is not a member. It's welcome to join though, the consortium says.
The background includes the point that Intel bought FGA supplier Altera for $16.7bn in December 2015, and has its own QPI (Quick Path Interconnect) technology, not to mention Silicon photonics.
It seems to us that OpenCAPI overlaps with the Gen-Z Consortium open memory fabric initiative. Both address the memory/storage-class memory interconnect area.
Gen-Z aims to provide a direct-attach, switched or fabric topology – a flexible, high-performance memory semantic fabric – to interconnect Compute (SoC memory), FPGA and GPU accelerators, pooled memory, and network/storage IO links.
OpenCAPI is meant to allow any microprocessor to attach to coherent user-level accelerators and I/O devices, and advanced memories accessible via read/write or user-level DMA semantics.
Then there is the CCIX (Cache Coherent Interconnect for Accelerators) group formed by AMD, ARM, Huawei, IBM, Mellanox, Qualcomm, and Xilinx (but not Nvidia or Intel) in May, to devise and make a cache coherent interconnect fabric linking multiple suppliers' CPUs, accelerators – FPGAs and GPUs – and network adapters so they can exchange data and share main memory.
Why are there three such overlapping efforts? The simple answer is that none of the members of these consortia want to let Intel own the CPU-memory-accelerator-network interface technology and so grab an even higher percentage of customer spend on servers, plus favor its own CPU and FPGA efforts.
They each and all think that the PCI interface is too slow and a higher-speed, lower-latency interconnect scheme is needed to let faster processors talk to faster accelerators, DRAM and the various storage-class memories emerging (think XPoint-type stuff), and RDMA-accessed external arrays.
Gen-Z consortium members are AMD, ARM, Broadcom, Cavium Inc, Cray, Dell EMC, HPE, Huawei, IBM, IDT, Lenovo, Mellanox, Micron, Microsemi, Red Hat, Samsung, Seagate, SK Hynix, Western Digital Corporation and Xilinx.
Google and Nvidia are the only OpenCAPI members not in the Gen-Z consortium.
Support for OpenCAPI
OpenCAPI declares: "Backed by a total of more than 30 leading technology companies, the three organizations welcome each other's announcements as part of a collaborative industry effort to create an open data center architecture for the future."
As an example of vendor support, here's Tom Eby, VP and GM of Micron's compute and networking business: "Because open standards present the best opportunity for rapid innovation, the OpenCAPI, Gen-Z, and CCIX consortiums are an important step in ensuring that developing architectures can quickly adapt to capitalize on the dramatic benefits provided by new memory technologies."
Here's Gaurav Singh, CCIX Chair: "The CCIX group of companies believe in the need to foster innovation in the industry. We welcome the efforts of and look forward to collaborating with the Gen-Z and OpenCAPI consortiums, which will further the development of key technologies that will define the data centers of tomorrow."
Completing the fulsome threesome's joint supportive comments is Kurtis Bowman, Gen-Z Consortium president: "The Gen-Z Consortium is committed to establishing an ecosystem where members, the broader industry, and customers work together to deliver robust, high-quality specifications that enable new data center architectures. The formation of these three new consortia (CCIX, OpenCAPI, and Gen-Z), backed by more than 30 industry-leading global companies, supports the premise that the data center of the future will require open standards. We look forward to collaborating with CCIX and OpenCAPI as this new ecosystem takes shape."
Key customers for any non-proprietary alternative to Intel fabric interconnect technology are:
- The server and hyperscaler suppliers: Cisco, Dell, Fujitsu, HDS, HPE, Huawei, IBM, InSpur, Lenovo, Oracle, Quanta and Supermicro on the server side.
- Businesses like Amazon, Apple, Facebook, Google and Microsoft on the hyperscale data center side.
- End users will basically buy what the server vendors adopt.
The three consortia, in a crude view, represent overlapping anti-Intel groups, wanting to constrain and limit chipzilla's ability to grab more of the server silicon component market to itself. It would be more hopeful for them if they could combine in a single CCIX, OpenCAPI, Gen-Z organization, but the individual members' concerns and technology preferences would make such an attempt unwieldy and difficult to bring to a successful conclusion. Hopefully, and nonetheless, they will try anyway.
Intel would rather face, we're sure, three partly or ill-coordinated and overlapping groups that collectively confuse interconnect-buying customers, and contrast the trios's confusing messages and timescales with its own, hopefully clearer and simpler message. The CPU-memory-network interconnect technology wars are on. ®