Preferred way to pass recognized sequences from Temporal Memory to a "higher HTM layer"?

Engineering NuPIC
,
1

Say I have trained a Temporal Memory instance with letters (encoded as SDRs) to distinguish sequences like “GEC” or “DEF” and predict the letter C after GE or the letter F after input of DE, something like it was seen in this one video from the HTM School series (High Order Memory vs. Single Order Memory). How can I use the recognized sequences as input in another Spatial Pooler or Temporal Memory?

My first guess would be some sort of SDR hashing: copy those SDRs in a row that make up the sequence and form a hash value, which you in turn encode as an SDR.

Another idea would be to let the Spatial Pooler solve this task by taking the sequence as input for the SP - which unfortunately has the problem that the sequences can be of different lengths.

1 Like
2

Hi,

IMO hashing is a bad idea. It is certainly not what the brain does. A problem with hashing an SDR is that the hash depends on the exact value of the SDR, so if even a single cell’s activity is different then the hash will be totally different. Hashes don’t account for the semantic similarities between their inputs.

This article describes a good solution to your problem.
(PDF) Learning Invariance From Transformation Sequences. Foldiak, 1991

I’ve personally tested this solution and you can see my results at the end of the video I recorded about it. In the video: I created a temporal memory (TM) and fed words into it, one letter at a time. Then I used this method to make an SDR that represents the current word, given only the output of the TM.

1 Like
3

Hi,
many many thanks! This experiment with Temporal Memory ( TM ), fed words and low-pass filter is very interesting. Do you have a script at hand showing how the experiment is set up?

4

Hi,
The experiment shown in that lecture uses my own implementation of an HTM, and I’ve got to warn you that it’s experimental (not production ready) and that I no longer maintain that implementation at all.

I wrote a report about that experiment, which explains in detail how I did it: sdr_algorithms/ascii_stability.pdf at master · ctrl-z-9000-times/sdr_algorithms · GitHub

And here is the experimental setup source-code: sdr_algorithms/ascii_stability.py at master · ctrl-z-9000-times/sdr_algorithms · GitHub

I hope this helps, if you have any other questions I’d be happy to hear them!

1 Like
5

hey thanks for sharing!

I tried running ascii_stability.py and got this:
from synapses import SynapseManager
ModuleNotFoundError: No module named ‘synapses’

In my cloned repo/directory I got this synapses.pyx file. Haven’t used cython before but I guess that I need this to make it work, right?

6

Yes, first you need to install and run cython.
The following command should work on linux:

./setup.py build_ext --build-lib `pwd`/../

If you’re on windows then you’ll need to replace the “`pwd`/…/” part with the directory that contains the sdr_algorithms folder.