# Symphony from Synapses: Neocortex as a Universal Dynamical Systems Modeller using Hierarchical Temporal Memory

Two great recent talks concerning the theory behind this paper from George Sugiharaâ€™s lab:

For HTM heads, this theory identifies what they call â€śsingularitiesâ€ť with high raw anomalies in the NuPIC sense, where prediction is impossible or highly sensitive to initial conditions.

1 Like

Fergal, thanks for the interesting and informative paper. Iâ€™ve only given it a quick read, but Iâ€™ll definitely digest it further.

To summarize (for my understanding), important features of one or many systems with many dynamic and unpredictable parameters can be reconstructed through observation of one parameter over time. This reconstruction is modeled by Takensâ€™ Theorem, which uses various delays from the time-series measured parameter to reconstruct properties of other parameters. The hypothesis is that neocortical structures like the Cortical Column use this principle to create a model of our dynamic world.

Iâ€™d also like to point to your blog writeups. They are really helpful:

Dave

Thanks @ddigiorg for taking a look and your kind remarks.

Yes, that is correct, but the claim is somewhat stronger than that in two ways.

First, Takensâ€™ Theorem states that the â€śmodelâ€ť to all mathematical intents and purposes is the system from which you get the time series (formally, the manifolds are diffeomorphic), which means that anything you use the model to do (prediction, anomaly detection, detection of nonstationarity, identification of regimes) will automatically be true of the real world system.

Takensâ€™ Theorem is a theorem (ie universally true, like Pythagorasâ€™) for specific classes of dynamical systems, but as Sugihara et al and many others show, it still holds in large part in almost all natural settings, even where the systems and the time series do not match the criteria.

The second point is illustrated by the examples given in the two talks above. To extract the parameters of the correct embeddings, scientists need to try different settings for the dimensionalities, lags, and choices of input variables in the time series, and choose the one which gives the best prediction scores or wins on some related measure, all involving observing how nearest neighbours stay close together as you change dimensionality.

HTM and the cortex do this automatically in L4 by projecting the inputs into the very high dimensional space of the TMâ€™s cellular SDRâ€™s, and using its learned coincidence detection to automatically lock on to the appropriate dimensionality, lag etc. By Temporally Pooling in L2/3 over highly predicted transitions, HTM models nonstationarity in the observed system (which these guys canâ€™t handle) and identifies the sequence of regimes the system is passing through.

As Sugihara says in the Q&A, he knows heâ€™s figured out the parameters he needs when his model does good predictions. This is exactly how HTM and the cortex do it, but the parameters are learned and the system need not be stationary as long as its regime trajectories can be learned.

1 Like

@fergalbyrne Once again I appreciate your rich explanation about how Takensâ€™ Theory applies to the cortex and HTM. Iâ€™ve never heard of concepts like dynamical systems, Takensâ€™ Theorum, and diffeomorphism so this is a really unusual perspective and quite interesting!

HTM and the cortex do this automatically in L4 by projecting the inputs into the very high dimensional space of the TMâ€™s cellular SDRâ€™s, and using its learned coincidence detection to automatically lock on to the appropriate dimensionality, lag etc. By Temporally Pooling in L2/3 over highly predicted transitions, HTM models nonstationarity in the observed system (which these guys canâ€™t handle) and identifies the sequence of regimes the system is passing through.

Hmmâ€¦ I think Iâ€™m having trouble understanding the relation between the current HTM theory architecture with the Takens Theorem. Other than my wildly unfamiliarity with these new concepts, a portion of my confusion stems from the functionality and interaction of layers. The layered architecture you explain in your paper seems a bit different from my understanding (unless your HTM theory is a bit different than Numentaâ€™s which might explain my confusion). However, I donâ€™t want to further derail the thread because I havenâ€™t had a chance to fully understand your paper. I apologies I canâ€™t contribute anything meaningful to the discussion!

At least for me Iâ€™ve identified a glaring gap in my knowledge. I really donâ€™t have a solid intuition about a cortical columnâ€™s architecture and itâ€™s holding me back. I think I might do a paper or detailed diagram myself to collect my thoughts and finally start digging into some neuroscience sources. I hope you donâ€™t mind me PMing you some questions in the coming weeks.

A side note (and attempt to contribute at least something to the conversation), I think at 3:18 and onward in this video may be indicating the L6 transformation current HTM theory. L6 transforms â€śwhatâ€ť and â€śwhereâ€ť inputs and L4 performs sensory-motor inference and prediction on that transformation. I could be wrong though, Iâ€™m not confident in my understanding just yet.

EDIT: For those who are unfamiliar with Dr. Sugiharaâ€™s work on dynamic modeling, I found an interesting article.