CSIRO orders 2,144 core, 174,720 GPU upgrade

The CSIRO has announced an upgrade to its GPU cluster to a hefty 134 nodes powered by 268 Intel E5-2650 processors and 390 NVidia Tesla 2050 GPU cards.

The 2,144 CPU core, 174,720 GPU core thumper will, according to the science agency’s head of Computation and Simulation Sciences Dr John Taylor, deliver up to 120 Teraflops of processing power, making it not only the fastest cluster in Australia, but could see the machine join the world’s top-100 list of high performance computers.

The cluster serves many of CSIRO’s research activities, Dr Taylor told The Register, including agricultural research, biomedical and bioengineering activities, mining research, environmental sciences and statistical modeling.

“Just about every area of CSIRO is able to make use of the cluster,” he said.

Since “hundreds of millions of dollars” worth of research activities are supported in one way or another by the cluster, Taylor said, its speed can easily justify the roughly $AU1 million price tag of the upgrade. “There’s a huge amount of leverage off these computer systems,” Taylor explained.

As an example, he said, a task such as analysing the output of an experiment at the Australian Synchotron in Melbourne might take three weeks prior to the upgrade. If something wasn’t right about the result, the researcher would have no option but to book another slot at the synchrotron.

“With our CT reconstruction code running on the GPU cluster, people can put the sample in, collect the data, and see the images in 15 minutes. If there’s something else to look at, they can adjust the experiments on the spot.

“So it makes the experience of going to the synchrotron more interactive, and improves the productivity by as much as 20 percent,” he said.

A key to getting the most out of the cluster has been to train the user base in how to code for the cluster. “You have to understand how your code works, how the data is being moved, how the instructions are being called – and make sure that matches the hardware hierarchy,” he explained.

Xenon Systems says its solution runs up to four GPUs per unit of rack space, and runs up to 8 gigatransfers per second, and uses the new FDR10 InfiniBand interconnect solution.

Xenon had worked with CSIRO on previous iterations of the cluster, helping it achieve 11th place on the list of the world’s greenest supercomputers last November. Since then, Taylor noted, other implementations had pushed it down a few spots to number 14 or 15.

Taylor said Xenon “has been very helpful, they’ve been a partner in helping get the cluster up and running, and keeping it optimal, and providing a good service to our scientists.” ®