I am implementing a simpler model of Temporal Memory as a protocol, for testing. I am hoping for some feedback.
- I have not implemented Spatial Pooler. Just temporal memory where each column is assigned (by me) to represent a particular item.
- It has only 5-6 columns and 20 cells per column.
- No segments.
- Threshold is 1, for distal connections. So only one connecting cell needs to be on to put the next cell in predictive state.
Otherwise, there is permanence and weights. A weight is connected if the permanence goes beyond a threshold.
Question 1: Would this work? Do you see a fundamental problem with this simple system?
There is a decision to be made: when the presynaptic neuron fires, and there is no predictive cell, all neurons in the postsynaptic column fire. How to choose the postsynaptic neuron to be potentiated? When there is a segment, the cell with the most active segment is chosen.
I have implemented this as follows:
- Choose the cell with the highest weight between pre- and post- synaptic neuron.
- If the connection is already established (permanence is above threshold) between pre- and post- cell, then strengthen the permanence further. Else,
- If the cell (pre- or post- synaptic) is used in some other connection (weights are already connected), then remove it, and go to the step 1.
- If cell is not used, then strengthen connection between pre- and post- cells.
Step 2 is introduced so that one cell is not shared between sequences as far as possible. I can also make this second rule more fuzzy - in step 2, the threshold used is lower than that for connection. For example, if permanence is above x, connection is established. If permanence is 0.9x, then connections are strengthened further, step 3 is not executed.
Question 2: Do you think this follows the principles of original HTM? Any comments?