It seems like LTD (long term depression) and LTP (long term potentiation) are the most common mechanism used by dendritic terminals to learn. I think that this mechanism is well understood by the neuroscience.
Nevertheless, it looks like the learning rules used by HTM is not following this. Especially LTD (which states that if the presynaptic neuron fires and the post-synaptic neuron don’t, AMPA receptors will be “reabsorbed” by the post-synaptic cell, which reduces the “permanence” of the synapse). In HTM this fact is mostly ignored. Perhaps distal punishment models somehow it. Nevertheless, in proximal dendrites there is no LTD at all. If my understating is correct, the rule used is the oposite: weaken the synapses of the cells that fires when the presynaptic cell don’t. This provides better results for SP/L2/… (there is a wider separation between SDRs) but lefts many unused synapses scattered across the system. In “scarce” memory conditions be able to prune such useless synapses might be interesting .
My question is: Why LTD is not used by HTM? I guess this is already proven by someone else. It seems like LTD has a high computational cost (whereas not for the biology since is a local process) but I don’t know if there is another hidden problem/s.
We certainly have LTP - that’s the main learning rule in the TM.
LTD is harder to get right. Any simple LTD rule will wipe out a lot of learned sequences that you may find useful later on. Cells in our models can make multiple predictions, and LTD would make it hard to maintain multiple predictions from the same cell. So how do we store memories for years without reinforcing them in between? That’s a research topic and hopefully we will understand this better as we put more of the cortical pieces together.
That said, we do have a proxy for LTD in the form of segment re-use. We typically limit the number of segments per cell and then have different ways to choose which segments to throw out when we need new ones. This isn’t LTD, but it is a practical way to achieve some of the same goals. I believe some form of this exists in the production (backtracking) implementation but not in the simplified “TemporalMemory” implementation.
@ycui yep. I was aware of it. I think that this translates to the punish synapses code, right? Nevertheless, biologically speaking LTD seem to be applied pervasively (no matter if the cell was predicted or not).
As @scott suggest, such punishment (if is too high or frequent) might kill useful concurrent “prediction paths”. The core issue one might think is if this is also the case of the biology. Intuitively LTD is equivalent to say that no cell will be shared by multiple prediction paths. In other words, the constant appearance of a sequence might “kill” another partially shared sequence. In summary, apparently there is a contradiction between parallel predictions and LTD… Probably segment reuse is enough for practical uses of TM. Nevertheless, this apparent contradiction is a bit itchy
Regards
vpuente
PD: Still in proximal dendrites (L4 an L2) might be useful (and not counterproductive, since there is no prediction) to perform LTD. Some simple experiments I did seem to suggest that.
We’ve tried many variations of SP learning rules over the years but I don’t remember if we tried LTD. It would be cool if you wanted to try out some experiments and post the results!
The problem I have is that I’m using my own implementation [I don’ have boost and use my own encoder]. I don’t know exactly how Nupic will react. I’ll try to do the same with Nupic.
Nevertheless, the thing I’m doing is just keep using the Nupic rules and apply LTD (with -10% of the learning rate) to the losers of the inhibition.
