Associative Memory via Predicitve Coding

I think of classical associative memory as pattern completion; part is provided as the key and the final product is the response. This is a simple spatial process. The input and output are all part of the same neural network system - there is no separate AM sub-system, this is built right into the fabric of the NN system.

When you extend this pattern matching/completion to include temporal content you have predictive coding.

The degree of temporal extent can vary. HTM matches to a single time step, as defined by the update cycle of the system. This HTM prediction is updated at each time step and can include the possibility of many possible pattern matches. The failure to match any of the stored patterns is considered a valuable indicator of surprise and IMHO had been the area of the greatest utility of HTM so far.

A pure Markov chain can predict much further into the future. Text auto-completion would be a good example of this in action. The pattern completes to several possible futures and is offered as the system output. Generally, the tree of offerings is pruned based on some sort of statistical weighting. The pattern completion process may be more specific as more input pattern is offered.

3 Likes