My analysis on why Temporal Memory prediction doesn't work on sequential data

I hope this isn’t going off topic (I think it is relevant to the topic), this particular goof may hint at a possible direction to explore for stabilizing the representations in a repeating sequence. If the system is allowed to make a smaller number of additional connections (beyond maxNewSynapseCount) for some of the current winning cells (some number of them above activationThreshold) to a random sampling of the winning cells from T-1 that it isn’t already connected with, then the representations would stabilize after the second time through the repeated sequence.

This would then lead to the case you mentioned of ambiguity for the C in ABCD vs XBCY. However, this implementation would result in duplicate cells in a sub-set of the C columns for the C in XBCY. One of the duplicates would be the same cell from ABCD and would be connected more weakly than the other duplicate which is unique to XBCY. The learning step could be tweaked to degrade the weaker of the two, which would eliminate the ambiguity.

It is an interesting idea. I’ll have to think it out further, and I’ll let you know what I learn from it.

1 Like