Understanding Intra-Region Connections and Temporal Context Requirements

[mh01223]

Hi, I’ve read a few HTM background papers, but there are still a couple of theoretical issues that are not completely clear to me.

(1) Am I correct in understanding that a cell within a region can connect to any other cell via multiple possible paths (multiple different synaptic connections between the dendritic segments themselves)? And this is why we cannot quantify an intra-region cell-connection via an assigned weight for each possible cell-connection? If this is true what purpose do multiple paths serve, given they all represent the same cell-connection?

(2) For a complex stream of data, would you not need a huge amount of cells for each column, given there are many different conditions that can precede a particular value?( ie. for a letter in the alphabet would you not need 25 cells for each column, to represent 25 possible contexts?)

Thanks!

[Paul_Lamb]

@mh01223

(1) Am I correct in understanding that a cell within a region can connect to any other cell via multiple possible paths (multiple different synaptic connections between the dendritic segments themselves)? And this is why we cannot quantify an intra-region cell-connection via an assigned weight for each possible cell-connection? If this is true what purpose do multiple paths serve, given they all represent the same cell-connection?

A given cell may follow another given cell in multiple different contexts. This is related to your second question:

@mh01223

For a complex stream of data, would you not need a huge amount of cells for each column, given there are many different conditions that can precede a particular value?( ie. for a letter in the alphabet would you not need 25 cells for each column, to represent 25 possible contexts?)

A couple of points to keep in mind:

1. Each input is represented by multiple minicolumns
2. Each cell can grow multiple distal dendrite segments that allow it to be used in many different contexts
3. Learning cells are chosen at random in each minicolumn from among the cells that have the fewest number of existing distal dendrites.
4. The activation threshold prevents a cell from becoming predictive if only a couple cells that a distal segment is connected to are active

Thus, there are many, many more ways to represent the same input in different contexts than just the number of cells per minicolumn, because the cells are shuffled and reused without breaching the activation threshold and causing false positives. When false positives do occur, the high noise tolerance property comes into play.

[Bitking]

@mh01223

(1) Am I correct in understanding that a cell within a region can connect to any other cell via multiple possible paths (multiple different synaptic connections between the dendritic segments themselves)?

No. The dendritic arbors are limited to a relatively short distance.
From what I have read - perhaps 0.5 mm. This works out to the nearest 220 or so mini-columns.

For more than you probably want to know about the wiring details:

Project : Full-layer V1 using HTM insights

I have been conversing with @gmirey for some time on this general topic. We endeavored to get a sense of the scale of the relative dimensional quantities of the temporal layer, inputs and cell/column spacing, number and range of the connections. You know - the basic facts of making a realistic model. Some of what was discussed was what is known about reach and density of the dendritic arbor in the “input” layer. We established the key factors would be: what is the spacing of the ascending cort…

[Paul_Lamb]

@Bitking

No. The dendritic arbors are limited to a relatively short distance.
From what I have read - perhaps 0.5 mm. This works out to the nearest 220 or so mini-columns.

Note that the current TM algorithm does not take this into account. Any cell may connect to any other cell, and cell A may connect to cell B on more than one distal dendrite segment (but not on the same one).

[Bitking]

Agreed - most of the current models are small enough that this has not been an issue.
As CPU/GPU power increases this will become more of an issue.

[mh01223]

Wow I didn’t realize they were being re-used; that’s very clever, thanks!