IBM boffins stuff 16 million-neuron chips into binary 'frog' brain
Cram 5.4 billion gates, million neurons in 28nm CMOS processor
Updated IBM researchers developing chips that mimic mortal brains say they've built a 4,096-core processor that simulates a million neurons.
The SyNAPSE silicon, fabricated by Samsung using a 28nm process, has 5.4 billion CMOS transistor gates, consumes 70mW of power, and uses a processor architecture completely unlike today's CPUs.
That's according to Big Blue, although a paper detailing the new package will be published in the Science journal on August 8. To test-drive the tech, IBM whacked 16 of the TrueNorth chips on one circuit board to model "sixteen million programmable neurons and four billion programmable synapses."
The neural-network project is funded by the US government's secret-squirrel military tech bureau DARPA: since 2008, Uncle Sam has plowed $53m into the chip team, which includes experts from Cornell Tech and iniLabs as well as IBM Research. In fact, it was Cornell Tech that came up with the asynchronous design of the processor.
"We foresee new generations of information technology systems – that complement today’s von Neumann machines – powered by an evolving ecosystem of systems, software, and services," said Dr Dharmendra S. Modha, chief scientist of IBM Research's Brain-Inspired Computing team, in a statement.
TrueNorth ... a brain-inspired chip architecture built from an interconnected network of lightweight neurosynaptic cores (Source: Esser, Andreopoulos, Appuswamy et al)
The focus at the moment is density and power – 20mW per cm2 – so the technology can be squeezed into pockets or tiled into huge fields to build "neurosynaptic supercomputers" with hundreds of trillions of synapses ... well, in theory anyway.
"These brain-inspired chips could transform mobility, via sensory and intelligent applications that can fit in the palm of your hand but without the need for Wi-Fi," added Dr Modha, suggesting gadgets shouldn't have to rely on cloud-based systems like Siri and Google Now to make complex decisions.
Professor Rajit Manohar of Cornell Tech chipped in: "After years of collaboration with IBM, we are now a step closer to building a computer similar to our brain."
'It's a bit of a monster chip'
IBM reckons the new silicon "has broken new ground in the field of brain-inspired computers," but it is not alone in developing hardware and software to mimic the human mind.
For example, Palm Pilot inventor and renegade neuroscientist Jeff Hawkins' approach is to model the neocortex, specifically how information enters the brain, is recorded as it filters through columns of neurons, and recalled from a hierarchy of memory. Meanwhile, boffins at Stanford are working on very low-power Neurocore chips, each of which models 65,536 neurons.
"The [IBM] TrueNorth chip announced today is an impressive integration of multiple neurosynaptic cores on a single chip," Steve Furber, a professor of computer engineering at the UK's University of Manchester, told The Register on Thursday.
"The neurosynaptic core looks pretty much like what they have been talking about for a couple of years, with its strengths and limitations, so what is new here is putting 4,096 cores on a single chip.
"It's a bit of a monster chip, at 4.3 sq cm on a 28nm process, and I'd be interested to know what yield they are getting, even allowing for the intrinsic fault-tolerance of the neural circuits."
Since 2011, Prof Furber, a co-designer of the original ARM processor chip in the 1980s, has been working on a project to ultimately model one billion neurons: SpiNNaker. This technology uses more conventional hardware – 18 32-bit ARM968 cores per chip – to build a massive parallel computer targeting neuroscience and robotics. Today, the system scales to 2,500 processors; a million-core machine built from 1,200 48-node boards would require 74kW at its peak [IEEE paper, May 2014].
"There is no conventional processor core in the IBM chip," Prof Furber said. "Unlike SpiNNaker, which uses ARM968 cores and implements neural and synaptic models in software, hence our flexibility, the IBM neurosynaptic core implements everything directly in hardware, hence their impressive density."
'All neuromorphic systems are trade-offs'
While Prof Furber described the TrueNorth silicon as impressive, given how dense it is, he noted the design uses "binary synapses, so most neural plasticity mechanisms can't be used," and "it only has 256 inputs per neuron, whereas the human brain averages 10,000."
Highlighting its strengths, the professor said IBM's chip is "digital, and matches the emulator model exactly – modulo yield faults – so you can develop the model on the emulator and then download it to the chip for real-time use."
"All neuromorphic systems are trade-offs," he added. "We are focussed on modelling and understanding biology, where our flexibility really counts; IBM is focussed on engineered applications, where efficiency and density probably count for more than biological fidelity."
This time last year, Big Blue emitted a new programming langauge for its SyNAPSE chip, which is to be expected as the silicon uses an unconventional processor architecture. The IT giant is still working on creating practical software for the thing.
The aim is to build a computer with ten billion neurons and a hundred-trillion synapses that consumes just one kilowatt of power and occupies less than two litres in volume.
That's two orders of magnitude down from what DARPA describes as a "human-level design", as you can see from this slide. Today, the technology represents 16 million neurons as opposed to the roughly 100 billion inhabiting each of our grey matter.
"16 million neurons is roughly the scale of a frog brain," Prof Furber quipped. "So, the IBM board may be able to catch a fly for its dinner."
A spokesperson for Big Blue was not available for comment at time of writing. You can catch more about the design of its custom processor in this paper, Cognitive Computing Systems: Algorithms and Applications for Networks of Neurosynaptic Cores, and here: A 45nm CMOS Neuromorphic Chip with a Scalable Architecture for Learning in Networks of Spiking Neurons. ®
Updated to add
After publication, Dr Dharmendra Modha got in touch with The Register to describe the custom architecture of the TrueNorth processors:
Although we are using a commercially available Samsung process, we have developed a custom brain-inspired architecture which is our key innovation.
Today's computers – for example, ARM cores – separate memory from computation via a bus. This requires a lot of energy and bandwidth to move data to-and-from the memory. Additionally, the processor is sequential, synchronous, and fault-prone.
In contrast, TrueNorth architecture is a network of neurosynaptic cores. Each neurosynaptic core tightly integrates memory and computation thus significantly reducing energy and bandwidth challenges. All cores are distributed and operate in parallel. Each core computes in an event-driven fashion and different cores communicate in an event-driven fashion.
Further, the architecture is tolerant to failure of cores. Furthermore, TrueNorth chips are equipped with an inter-chip off-chip interface allowing them to tile. Tiling means that the architecture is designed to be scalable from the ground-up.