First of all, I hope that my analysis is flawed, and someone points out what I’ve got wrong. I like a lot Jeff Hawkings’ theory on how the brains work, and really want to see HTM solving real-world problems! However, I think I’ve gained enough knowledge about HTM through reading almost all papers published by Numenta, stepping through the NuPIC’s code, and playing with sanity-nupic visualization tool to share my observations.
It seems that the main problem with TM is that it can’t recognize when an event (i.e., “input pattern”) occurs again in the same context. I’ll use a very simple input sequence: “A,B,C,A,B,C,A,B,C,…”. TM in its form presented in papers can’t learn to predict that “B” occurs after “A”, “C” after “B”, and “A” after “C”. Assume we use the following parameters:
- columnCount: 6,
- cellPerColumn: 16,
- initialPerm: 0.51,
- connectedPerm: 0.50 (a synapse becomes connected as soon as it is created),
- maxSegmentsPerCell: 1,
- maxSynapsesPerSegment: 1,
- activationThreshold: 1.
Also assume that the spatial pooler assigns columns 1,2 to “A”, columns 3,4 to “B”, and columns 5,6 to “C”. Here’s a visualization of what happens to TM cells on each step through the input sequence:
In steps 1-4, TM bursts the columns corresponding to the input events (shown in red).
In step 1, cells #1 in each column are randomly chosen to represent A’s context.
In step 2, cell #6 in column #3 and cell #3 in column #4 are randomly chosen to grow a single synapse on each of them to the cells that were chosen to represent “A” (in this example, I don’t talk about segments since there is 1 synapse per segment).
The same is done in steps 3 and 4: TM randomly chooses a cell in a column and grows a synapse on it to a cell chosen to represent the previous input.
In step 5, cell #6 in column #3 and cell #3 in column #4 were predicted since both cells #0 in columns #1,2 were active in the previous step (as these columns bursted). So, only these cells are activated.
Similarly, events “C” and “A” are correctly predicted in steps 6-7.
However, in step 8, “B” is unexpected since its representative cells #6 (column #3) and #3 (column #4) were waiting for cells #1 in columns #1 and #2 to get activated. Instead, cell #13 in column #1 and cell #11 in
column #2 were active. Thus, both columns #3 and #4 burst.
This scenario repeats continously: 3 correct predictions followed by a misprediction.
As I figured out, TM needs that the user specifies when to “reset” an input sequence, that is, it needs to know when the context of the first event occurs again. I suppose that’s why all the datasets used in NAB encode the date as a part of the input. For example, in the “hotgym” dataset used in the HTM School youtube videos to teach how TM predicts time series, the context of the energy consumption is defined by the day of the week and the time of the day. Correct me if I’m wrong about this.
P.S. I’m aware of the backtracking_TM used by default in sanity-nupic (I couldn’t even find a value for the “temporalImp” parameter that would force NuPIC to use the original TM class defined in temporal_memory.py). The comment to the _inferBacktrack() method in backtracking_TM.py kind of explains why it’s used. However, I don’t see how backtracking can help with the issue I described above. Also, why it is implemented (and used by default) if it’s not a part of the “official” HTM model and is not explained in papers?