We need new hardware

[adam]

Hi, I definitely agree with you, we need new hardware!
However I think we are very far from what we actually need.

If we look to the brain, we see the following: Processing capacity and memory are mixed together. They are not separated like in our today’s computers!
Information does not have to travel over a limited bus. It is where it is needed! The algorithms are working locally, not globally.

Hand-wavy analysis of HTM on FPGA synthesiser report

I managed to scrape some time together to compile one of Etaler’s OpenCL kernel into FPGA. These compilation takes hours on a large server. So I only have little to show. Hopefully some one in the future will find this post useful. Setup I’m using Altera/Intel’s OpenCL HLS Compiler to compile OpenCL into FPGA. And no optimization is attempted on making the kernels suitable for FPGA; they are used as-is. Also I’m targeting the quite old DE5-Net board, but it is the largest and the fastest I have…

In this post the argument is made, that a external DDR3 memory needs to be connected to the FPGA: That makes total sense! They need to connect that memory to the FPGA. But as they do so, they immediately get the problem of having a limited memory bandwidth.

More brain like would be to mix the memory and the computing power. Then there is no memory bus and the information does not need to be transferred.

This would also solve another problem, PCs currently have: Heat! Heat is generated when fast clocking operations are executed (Assuming that the static power consumption is not the source of heat).
If we mix computing power and memory, as well as run the algorithms locally, then we would get two things:

• more space between the parts that get hot (As memory parts usually stay cold)
• less parts that need to have a fast clock (As we could disable clocking for unused parts)

But until we get there I think we have to stick to our memory bus, inherent in all PCs/micro-controllers/etc.

Regarding your thoughts:

• If only memory is the limiting factor, then we can use typical server hardware. Computers with 1TB of RAM are already available (Even though costly).
• Projects like SPiNNaker (Research Groups: APT - Advanced Processor Technologies (School of Computer Science - The University of Manchester)) push in the direction of having smaller processing units, more densely connected. This allows to have a MUCH bigger memory bandwidth. However they are still bound by their architecture.

Even though the hardware is interesting, I am not convinced, that we actually need those big simulations to discover new things.
Small scale simulations usually do the trick and even if such small simulations need to run a week on our old-style PCs, that is still OK.

Adam