HTM Mini-Columns into Hexagonal Grids!

Conway’s Life? That is an interesting insight.

I am not aware of any connection.

On the other hand, I have never thought of looking either.

Cellular automata, of which Conway’s thing is an instance, capture some essential feature of computation in an interconnected environment. Even with a 1D automaton, some very simple update rules are unexpectedly able to produce “complexity”… chaos, randomness ; and some are provably turing-complete. This has been studied by Stephen Wolfram (author of Mathematica, Wolfram alpha…) and explained in his “A new kind of science” book.

(I own a hard cover at home, this is cool stuff…)

3 Likes

Hmmm interesting, and currently half way through Calvins book.

Is any one interested in implementing such an approach on top of what is present in NuPic or would that prove to difficult?

2 Likes

The Calvin cells implemented in L2/3 would work well to do the spatial sparsing function. This could be dropped into the standard HTM model with very little work. The impact on binding/tiling is a rather pleasant side effect at that point.

Hi, @Bitking

Unfortunately, I often get lost in your explanations. Am I getting it right that cells grids actually offer a solution to the combination part of the binding problem and not to the segregation part of it?

Thanks,
Dmitry

Yes, you do have this correct.
You are also correct that I need a good editor.

Bitking can i clarify something with you.

The first and most important for me atm is how to treat the input that each mini-column / L2/3 cell would receive. In the spatial pooler implementation within a 2D input space (lets say MNIST) each mini column has a receptive field value (as it is an 2D array this is I guess a square around its centre). Within this receptive field (ie offsets from column index etc) a random number of input cells / array indices are chosen to be seen as connected, around 2% or however the original algorithm determines it.

If, as can be done already our spatial pooler outputs a 2D array of bits (i.e an 2D SDR), do you see each L2/3 cell / minicolumn as simply being a single / one to one mapping to the input SDR, or does each L2/3 cell also have a receptive field in which it can connect to multiple inputs.

This ignores the 0.5m dendrite ‘halo’ element of the hex grid activations.

I’m just curious to see what the best and most plausible approach to feeding in the input from the SP to a calvin layer / L2/3 layer would be.

I’ve been looking at: Project : Full-layer V1 using HTM insights to get an idea of the ‘dendritic extent’ as gimery puts it, but wasn’t sure how to handle the actual input part.

Cheers

Repeating a key point: all of this is describing what is happening in the L2/3 layer. At this level all we are working with is patterns and NO predictive memory. The L2/3 is also the layer that talks with other maps though the output axons of the L2/3 cell bodies. Likewise, the sensory inputs and rising axons are projected upwards through all layer terminating in the dense mat of L1.

The projecting axons lateral branches do make connections with inhibitory inter-neurons not shown here.

I am certain that there are connections up and down the minicolumn between the layers but I am not ready to state exactly how these connections work. These connections are key to describing the relationship between pattern recognition and temporal prediction. Working out these rules would be key to understanding the training rules of predictive memory.

Now on to the parts relative to hex-grid theory …

If you look at my entry on the Project : Full-layer V1 using HTM insights post, #34 I put figures on the sizes of the various elements for minicolumn spacing.

Each cell in the mini-column has a few dendrites and each one is at least single SDR and maybe a few.


Referring to this picture from that post - each blue circle is a single mini-column. This is 100 or so cell bodies. Each cell body has it’s own dendrites - say 10 or so for a nice round number.

It turns out that the rising projecting axons are also spaced on this 30 µm spacing so you can assume that each blue circle also has a rising axon bundle.

The large black circle is the reach of the dendrites (+/- 250 µm, or 500 µm total) for the minicolumn in the center of the diagram. That gives the dendrites in each cell in each microcolum access to about 200 or so rising axon clusters. This can be thought of as a “receptive field” of the blue blue mini-columns and rising axons within this black circle for this minicolumn. This is all repeated for the next minicolumn. This is the repeating structure for all minicolumns in the cortex.

In this paper Horizontal Synaptic Connections in Monkey Prefrontal Cortex the lateral connections from the L2/3 cells are given as an average about about 500 µm.


The black beam in this picture is the long distance lateral connection between the two minicolumns so that the two minicolumn “receptive fields” connected by this link of the the hex grid covers the space with very little overlap and very little missed space.

When you factor in the fact that there are several long distance lateral connections emanating in all directions from each cell in the minicolumn you can see that the possibilities to form space covering hex grids is very large. Since these connections from any given cell do not fall on strict angles or length the individual cells can form hex grids with different angle, spacing and phasing.

So recapping, ever minicolumn has 100 cells that each have 10 dendrites, each potentially forming at least one SDR, possibly more. (at least 1000 SDRs per minicolumn)

Each Dendrite, if it went in a straight line from from the cell body passes at least 7 axon projection clusters and with branching probably many more. This means that the area around each minicolum is densely sampled with about 1000 branching dendrites which should end up sampling every rising axon cluster within reach of the dendrites.

The lateral connection link these minicolumns so that if they are responding to a learned pattern, even though it is sampled relatively sparsely, all the space in the resonating hex grid pattern is being sampled and bound together into a single larger unique pattern.

Here is a drawing relating the idealized concept to the messy biological bits.

Also see this post for more on the formation of larger patterns:

1 Like

To directly answer this key point - the hex grid pattern IS the spatial pooler. I see this as a direct replacement for the spatial pooler in HTM theory.

The point of the pooler is to collect activations and sparsify them; the Hex-grid is a naturally sparse activation.

Hmm I guess I got my assumptions wrong.

My interpretation was that the formation / spacing of the main hex grid forming mini columns was as a result of the 0.5mm dendrite ring or halo, exciting other mini columns on this periphery. If by coincidence 2 or more other mini columns are receiving the some of the input from the same SDR and their dendritic ring happens to overlap, then this would lead to overlap of excitation causing more mini columns to become entrained and activated.

So apologies for the poor drawing, my iPad skills know only so much. So I originally thought as hopefully shown in the bottom part that a mini column,excites its dendrites (black ring) and by luck another mini column around the circumference of this ring is also being activated by the input SDR then it to would activate its dendrite ‘ring’ (red). This would lead to an overlap between the black and red dendrite rings and cause local excitation of other mini columns where this occurs. Leading to these min columns to excite and for the process to repeat and the hex grid pattern to grow as Calvin would say.

At the top is just an example using Gimerys, Hexagonal array layout. In this example because the distance is fixed (i shortened it for ease of drawing) you can get a number of hex grid patterns to emerge, in different orientations, but not scales or possibly phases. It all depends on where along the dendrite circumference the 2nd mini columns happens to be.

Looking over at your post I assume that it is the ‘Lateral axonal projection’ that gives rise to the hexagonal spacing and also is the cause of activation / recruitment of other mini columns?.

My original intention was to follow the previous approach. If I was to attempt to describe it programatically then it might go something like this.

SDR for SP provided as input to L2/3 layer -> 1-1maping of mini columns to SDR, ie each small circle is linked with a single SDR ‘bit’ (though you could have a receptive field if you wanted) -> For each SDR bit that is active you excite that mini column -> for each active mini column in L2/3 layer you go through dendrite ring and excite mini columns on their circumference -> for each active minicoumn in L2/3 layer you also perform local inhibition.

Because unlike the standard SP / TM approach this requires some degree of temporal element i.e repeat this process several times (ie every 10hz equivalent) until it settles on a hex grid.

This was just a very crude first glance. But looks like it is probably inaccurate.

If you have any recommendations or corrections I would appreciate them.

Regards

Momiji

1 Like

I am a bit sorry that my attempts to describe what I am thinking are hard to understand. It all seems so clear and logical in my head but from what you are saying I am doing a terribly job in describing what I am thinking.

I hope that the additional drawings in post #62 above are helpful.

I envision that the competition for for hex-grid formation runs at the gamma rate (40 Hz) and is resolved for each alpha cycle (10 Hz).

The output from this alpha rate update is what is communicated to maps attached by projecting axons from L2/3.

Oh don’t be sorry, lol I would be nowhere without your descriptions and images. It is just myself coming at this with an amalgam of different posts and readings from Calvins book, getting confused. Unfortunately I have not had as long a history in neuroscience to fix in my mind the terms and structures so I can get a bit confused with what I see. In particular what parts are receiving input from say step 1 to what parts are outputting into step 2 so to speak.

Kind Regards

Momiji

1 Like

I have been thinking about Calvin tiles for a very long time; how they might work and the neural substrates that they depend on. I was reviewing some old notes on the Braddick & Sleigh book “Physical and Biological Processing of Images” this evening and found this note.

Title page of the notes with a date in the margin:

And this comment in my notes:

Sorry about the sloppy handwriting - this was me thinking aloud as I read the text.

Fascinating collection of material; we took note of Calvin’s configuration in particular multiple points of convergence surrounding a central point (fig. A), as opposed to the central point and convergence being one in the same as in (fig. B).
abf25d83aa44da0c32700608400d464f8d910e8f

While those are possible connections they are not the correct distance away.
The lateral connections are usually a fixed distance apart.
You should like them - they are natural triangles.

More details on the geometry here:

1 Like

Understandable that a distance has been established with which to inform the geometry. It may be a surprising finding for us and we’ll be taking a look at our model to see how Calvin’s arrangement might correspond, but whether or not we revise our model to accommodate for Calvin’s arrangement may be another matter as our results so far have been overwhelmingly positive.

I’m quite late for the party, but lateral connections are mGluR, right? (efference copies of L6 descending axons)[1]. Those “nasty” synapses are really “interesting”: integrate (sum) over time, are really persistent and only have modulatory effects on proximal dendrites. My understanding is that the drivers (iGluR) are 10x more powerful.

[1] C. C. Lee and S. M. Sherman, “Modulator property of the intrinsic cortical projection from layer 6 to layer 4,” Front. Syst. Neurosci. , vol. 3, no. February, pp. 1–5, 2009.

How about the layer 2/3?
These are the ones that most concern me for grid forming behavior.

Also, do you know how these projection axons interact with the surrounding inhibitory interneurons?
This question is addressed to both layers 2/3 and 6.

Those synapses are in L4 PC proximal dendrites. I.e. modulates cortical column “input”. I guess are who incline the balance to who are the inhibition winners. The axons are really dense. According to this review [1] are a big contributor to the column input (at least for V1). Since the shape of the L6 axons is hexagon-to-hexagon, I thought that where your “lateral” inputs. Sorry about that.

[1] T. Binzegger, “A Quantitative Map of the Circuit of Cat Primary Visual Cortex,” J. Neurosci. , vol. 24, no. 39, pp. 8441–8453, 2004.

1 Like