I found that TemporalMemory runs slower and slower while training. Eventually becoming so slow that it is impartial to use. For example, taking 10 seconds to infer 1000 times after extensive training. At least 100 times slower than initially. Is there a solution?

[image] breznak: TP had an option to do synapseDecay - eventually destroy a synapse if unused for a longer period of time, that is actually a biologically correct behavior. Does TM have it? If not, we can add that and see how it helps. I’m not aware that one exists in NuPIC.core’s TemporalMe…

[image] marty1885: I’m not aware that one exists in NuPIC.core’s TemporalMemory. That sounds like a good idea. Thanks! I knew it existed in numenta/nupic (py) repository in the old temporal pooler (TP class). TP is not there anymore, was it renamed, or completely removed? It’s c++ counterpar…

I have not run you example yet, but from the description of the problem: it sounds like your TM is going to have 100% anomaly. Between the random input data and the fact that isn’t enough activity to trigger a dendrite prediction. (64 inputs X 10% sparsity = 6 active mini-columns) Bursting mini-c…

If I remember correctly. Cells4 is the non biologically possible TemporalPooler while TemporalMemory is the biological possible one. [marty@zack algorithms]$ ls TemporalMemory.hpp TemporalMemory.hpp [marty@zack algorithms]$ cat TemporalMemory.hpp | grep decay #nothing [marty@zack algorithms]$ cat …

:neutral_face: Sorry about the misdirection about Cells4.cpp, that was advice I gave someone who was trying to do the same thing with the old “Backtracking TM”. For the new TM you guys are on the right track.

No prob. Thinking about it, the synapse / segment decaying would be nice to have in TM. Although I was a bit unclear too, decay unlearns a synapse when not used, in addition to the current unlearning when mismatched. The new thing would be pruning/reusing of the old synapses, segments.

[image] breznak: TP had an option to do synapseDecay - eventually destroy a synapse if unused for a longer period of time, that is actually a biologically correct behavior. The way I have implemented this is by tagging each synapse with last used timestep, and then I only need to apply the de…

I only need to apply the decay when a synapse is next used (and can remove it at that time Yes, on the other hand you keep synapses created only once, likely the culprit observed here. I’m thinking if decay could be a separate thread visiting synapses and lowering them, by the timestamp as you s…

I’m not sure about NuPIC specifically, but my initial interpretation of extraneous synapses causing TM to run slower over time (versus just consuming more memory) is that the algorithm is iterating over all synapses somewhere. HTM can be implemented without doing this. Instead of sampling from the…

… Then you can do LTD as biology does: if a cell fires but a postsynaptic connected cell don’t, you apply a tiny decrement in the permanence of the synapse. That way some unused synapses will go away progressively. LTD, at least from a biological perspective (i.e. activation of calcium-dependent ph…

TemporalMemory running very slow after long training

Engineering

Bitking February 5, 2019, 6:45pm 34

Or you can take this a step further and have a nightly maintenance phase where you shrink all synapses!

Whitepaper: Avoid extra connections

Topic		Replies	Views
Suggest change to Sequence Memory algorithm NuPIC sequence-memory	13	948	May 15, 2019
My analysis on why Temporal Memory prediction doesn't work on sequential data Numenta Theory sequence-memory	58	7640	February 2, 2020
The Principle of Temporal Memory Numenta Theory	21	852	April 25, 2021
Some details murky on temporal implementation Engineering	10	1408	September 6, 2017
Project to compare mraptor's bbHTM to biology Engineering	21	2153	June 22, 2016

TemporalMemory running very slow after long training

Related topics