Hi all, i am new to this concept and also excited to learn this new theory,can anyone explain difference between spiking neural networks and htm.
tl;dr - Spiking Neural Networks are lower level and are more accurate compared to HTM at simulating real neurons. But also a lot more computationally expensive.
In my opinion, on a higher level. Spiking Neural Networks simulates how every neurons works, connectes and responses accurately at the electric level in every moment (ex: every 1us). And the learning is generally done using the Spike-Timing-Dependent-Plasicity method. While the simulation is accurate and delivers promising results. SNN is also very computationally expensive that you need a specially designed super computer to run a miniature brain in real-time. HTM on the other hand attempts to model the Neocortex as a whole and ingoring the unimportant properties of neurons. HTM is a less accurate simulation of the brain, but also a lot faster. We hope that the model HTM builds will eventually be enough to create intelligence.
Let us know if you what to know the difference at a lower level. It’s going to be a long post!
Something I ignored, when I worked with SNN was, that dendrites play an active part in the signal processing (are non-linear). SNN equations have been developed by building a model of the cell membrane of a neuron. Therefore they can model very accurately, how the voltage and other parameters in the neuron change.
There are two different approaches, how SNN can be used in models:
On one hand, people try to build very accurate and complex models of neurons, where they simulate the whole structure (including the branching of dendrites) of the neuron. This approach allows to have a very accurate models of neurons. The price is, that these models are very very computationally expensive!
Another approach is to simulate just a single neuron (speaking in terms of the equations: just a single membrane point for each neuron). This is of course much less computationally expensive, however the possibility to integrate the non-linearity of dendrites is not possible any more. For these type of SNN there exist a lot of different hardware accelerators (like SpiNNaker, introduced in the post before).
It might be enough for some research to just simulate a few neurons with high accuracy, or to simulate much more neurons at a lower accuracy. But because we do not have unlimited computational power, we must build our models so that only necessary concepts are built into the algorithm. But the question is, what is a necessary concept?
HTM does not try to model every ion-channel of the cell membrane and so can not give you information about the voltage of cell. So HTM ignores some biological implementation details to save computational effort. But HTM is aware of important concepts and tries for example to model the non-linearity of dendrites.
thanks for the reply yes, i want to know the diffference at lower level and also i want to know about the similarity between the snn and htm, also which one exactly mimic the brain structure and functionality.
I think that SNN and HTM do different things; they work at different levels of representation.
SNN is more like sub-atomic theory, very correct with very little application in day-to-day engineering.
Common physics like Newtons laws of motion tell us very little about sub-atomic theory but are very helpful in building bridges and making airplanes that fly. HTM is more about this level of representation.
HTM does ignore some of what cells do as it is not considered key features of the computation that is performed. HTM captures certain useful subsets of cell behavior in a way that allows someone to test ideas about how the brain might be working.
SNN may tell us a great deal about how a small group of cells interact but is currently not useful for directly building larger models of brain behavior. The computer resources required are simply far beyond the state of the art. Even with the simplifications that models like HTM offer we are barely able to model a few square millimeters of brain tissue. SNN computing would require something like two order of magnitude of greater computing power to do the same task.
SNN does provide valuable clues on general guidelines for constructing these larger models. As we learn new things about the lower levels of how the cells work this information is used to adjust the way that higher level models work.
thanks for the reply
There are some spiking neuron models that are quite inexpensive computationally, iirc.
I also think also alternatively a cellular automata like spiking model might be functionally adequate whilst being relatively inexpensive too.
I concur. I am a total beginner at HTM. I am First reading as much as I can about HTM before throwing myself into coding. There is no doubt HTM is a great idea. I wouldn’t be here if I thought it wasn’t. Interestingly, it is neither DL, nor a classical biophysical (plausible) model. But I am still in a struggle thinking if not using spike neurons is really the way to go. On that respect, I think Neural Engineering Framework is ahead. But HTM seems conceptually a more complete theory. Well… It would be great finding a paper comparing spike NNs with HTM over the exact same task. Does anyone suggest a paper or text about it?
I don’t have a comparison paper but I’d highly recommend you check out HTM School if you haven’t already:
Once you really understand the mechanisms of HTM I think you’ll easily see what separates it.