Question about linking individual TMs

Hi, relatively new to HTM here. I’ve been building my own code library as a way to really learn the ins and outs of HTM. So far I have working encoders, poolers and temporal memory. Now I want to move on and build an actual hierarchy with multiple separate TMs. I’m wondering about how exactly to connect separate sections that are at the same level within the hierarchy. Would it make more sense to add connections similar to the axial connections that allow cells to make predictions? Or connections between mini-columns, which add to the total overlap score of the proximal inputs rather than between cells?

I appreciate any tips, or any pointers to relevant papers that can help! Thanks!

2 Likes

We’ve thought about this too.

There’s a variety of ways you can do it. You can make the inter-module connections be either distal dendrites which create the predictive state for neurons, or you can make them proximal dendrites, which simply add to active state. You could also make them apical dendrites which would activate according to some other rule.

However, the big question is what is the order of execution of these laterally connected modules? You now have a recurrent connection and you need to figure out some way to step them in a stable manner.

In the neuroscience theory, laterally connected modules are running until they find some stable state. Maybe that could work, but I don’t know if a stable state is guaranteed.

Another thing you could do is put a delay on the interconnected dendrites so that module A at time t is receiving the distal inputs from module B at time t-1. This ensures you don’t have synchronization issues, but you aren’t guaranteed to operate on the same data at time t.

It’s still an open question and something we’ve been experimenting on.

2 Likes

I’d definitely recommend looking at this module, and the repo containing it:

It demonstrates how they use their Network API to connect different regions with their larger cortical column module. There are different types of connection regions can have to each other. For instance the output of one region can be used to depolarize another (input to its TM) or activate another (input to its SP). These interdependencies between regions align with the roles each region is thought to play. For instance Layer 6 is though to represent location, and it has a depolarizing effect on Layer 4.

Also @jacobeverist is right, the order of execution is a huge part of it. That aspect is handled in the module using Phases.

3 Likes

Thanks @jacobeverist and @sheiser1, that is helpful! I have a followup question.

Suppose I use proximal dendrites to influence activity levels across the two SPs. Putting aside the recurrence issue, my difficulty lies in determining how much influence the inter-SP proximal connections have vs. the external proximal connections. Does it make sense to simply add them together, so that a minicolumn becomes active if the total input is sufficient? Or would it be better to consider the external/internal connections separately, each with its own overlap threshold, and a minicolumn becomes active either if (A) its external proximal overlap score is high enough or (B) its inter-SP proximal overlap score is high enough.

The second option makes more sense to me in situations where you want it to be possible for an SP to light up with activity despite getting no external input at all, if its partner got an input and became active.

Do you have any thoughts on this issue?

Thanks again!

Yes. :slight_smile:

Let us know how it works out.

I’m not sure what you mean here by “inter-SP connections” versus “external proximal connections”.

1 Like