HTM Mini-Columns into Hexagonal Grids!

gmirey · April 5, 2018, 9:03am

Oh, I have

About shoe-horning : Yeah well. As I said that’s more of a feeling than an argument that he’s wrong. I guess any top-down approach which also cares about the bottom-up side has to shoehorn things somewhere at edges before the whole picture is crystal clear. However since we’re both discussing things here on this forum, I assume we have a gut feeling that Numenta’s broad direction and/or methodology feels right. For me is trying to work from the initial insight that we’re predictive machines who’ll try to discover any structure in whatever input flow we get. I cannot prove it right, but it sure does not sound dissonant to my experience either, and it seems to point towards an explanation while trying to stay consistent with biology findings. An inner-darwin mechanism, on the other side, I have no clue whether it is a necessary requirement. This does not make this wrong either, but I wouldn’t bet my life on it.
So were he to see/feel/intuit/have proof of five over the six “ingredients”, then see a similarity with a darwin model, then try to get a grasp on the missing one, okay. Even if the sixth had required a little edge-rounding. But the book certainly reads like he saw the shiny hammer well before that 5/6 mark.

Granted, he has a life time exposition to these considerations, a bright, well-working brain, and this work shall not be dismissed on the basis that a novice like myself does not understand it. Very true. I’m not dismissing it per se. In fact the best I can do to try and understand, however rough this may sound, is to assault everywhere I do not understand until convinced that it holds.

From your linked post above, I’ve barely browsed ref. [8] and this seems well over my head already. However what comes after the burst does seem like a wave indeed.

pondering pondering

What I still do not get is the relationship you see between “the Moser Grid findings” and Calvin’s grid patterns. Other than the fact they both involve the hex lattice (which is in itself an optimal form for a lot of things, not necessarily intrinsic to brains or navigation). I mean in the very HTM school video you’re referring to, Matt makes it clear that those grid cells spike at fixed locations in the environment’s space, but this has nothing to do with their own layout in the cortex topology.

Sorry to go heavy on that same question twice, when you already took the time to try and answer me, but I’m really confused here.

Regards,
Guillaume.

gmirey · April 5, 2018, 10:24am

Me again, sorry.
I’m reading “Network mechanism of grid cells”. Haven’t finished, but wanted to let you know. Should have read it before, it seems. Here too they’re proposing that same correlation between environmental-grid-responses and internal-layout-“gridness”. I’m well open to the possibility that there is such a correlation, although I still cannot infer why.

Bitking · April 5, 2018, 1:05pm

The parts of Calvin’s book that are central for me are just -

the neural mechanism for grid formation.
A simple, biologically plausible, solution for the binding problem.
the level of thinking once you realize that grids do exist.
This is a big one or me. We go from a particular behavior (grid forming) to cooperative/competitive learning and what happens when competing population are perceiving some pattern. It fills a conceptual space somewhere between individual cells and whole maps. This is a new level of representation that we can discuss - It may not be Darwinism but how does it work?

My takeaway: There is a higher level of organization that can be tested and may have some new predictive power.

When I first read Calvin it was just an interesting idea: a sort of alphabet and grammar for coding. Please keep in mind that when Calvin proposed this nobody have ever seen this - it was just a prediction. When I read about the Moser work this suddenly got real - some actual real-world information coded just as Calvin predicted it would be.

I put this at about the same level as finding that DNA and the 4 letter alphabet is how genes code data.

gmirey · April 5, 2018, 1:41pm

I’m erring through papers. Currently jumped from Continuous attractor network models of grid cell firing based on excitatory–inhibitory interactions to its referenced Noise promotes independent control of gamma oscillations and grid firing within recurrent attractor networks.
What I find interesting in those two is the mention that “not all” of the cells of the precise type within the precise layer in which they find grid patterns, do actually arrange into grids. And that some bright people start to find models where indeed some cells do, and some don’t.

Each paper I read actually does a good job at hinting how much further knowledge I lack.

So I don’t know about an “all-in” for cortical grid patterns. But those grids are definitely something worth exploring.

Thanks again @bitking

Bitking · April 5, 2018, 2:30pm

As you wade through the sea of papers please keep in mind that not all areas of the brain are grid-forming.
I see that there is a progression:

Early sensory areas where “edges” or transitions are sensed and sharpened
Intermediate areas where the sensed streams are rearranged to distribute them over much wider topographical areas.
Hub areas where the “abstracted” contents of the associated lobe are communicated to other areas. This is where I see grid-forming cells. The grid-forming cells are also the level II/III cells that mainly project to other cortical areas. Other cells in the area may have other important functions but these are local functions.

The genetically defined parameters of individual cells control the relative lengths and targets of dendrites and axons to define the behavior of individual cells. This, in turn, results in the “processing characteristics” of the area.

gmirey · April 7, 2018, 8:20am

Things have not clicked into place in my mind yet… far from it, but I’m beginning to perceive part of what drives your insights.

good catch all this

Bitking · April 9, 2018, 4:10pm

Just following up on your note:

Bitking · May 16, 2018, 7:33pm

Papers offering support for William Calvin’s grid theory.

rhyolight · May 17, 2018, 10:26am

A post was split to a new topic: Posters summary for the Grid cells meeting in UCL

Bitking · June 20, 2018, 2:39am

I did a clean-up pass to add new material and clarify the relationship between the hex-grid-forming aspect vs. Moser spatial grid representations.

yannc · July 13, 2018, 9:52am

Hi there,

I’m currently reading The cerebral code from Calvin and see that hexagonal lattices might tend to spatially propagate with different inhibition mechanisms that seems to allow a better consistency of the representation. This propagation/inhibition “rules” made me think about Conway’s life game, would there be any related article about this similarity, if ever there is one?

thanks

Bitking · July 13, 2018, 3:38pm

Conway’s Life? That is an interesting insight.

I am not aware of any connection.

On the other hand, I have never thought of looking either.

gmirey · July 15, 2018, 11:18am

Cellular automata, of which Conway’s thing is an instance, capture some essential feature of computation in an interconnected environment. Even with a 1D automaton, some very simple update rules are unexpectedly able to produce “complexity”… chaos, randomness ; and some are provably turing-complete. This has been studied by Stephen Wolfram (author of Mathematica, Wolfram alpha…) and explained in his “A new kind of science” book.

(I own a hard cover at home, this is cool stuff…)

momiji · July 25, 2018, 2:48pm

Hmmm interesting, and currently half way through Calvins book.

Is any one interested in implementing such an approach on top of what is present in NuPic or would that prove to difficult?

Bitking · July 25, 2018, 3:25pm

The Calvin cells implemented in L2/3 would work well to do the spatial sparsing function. This could be dropped into the standard HTM model with very little work. The impact on binding/tiling is a rather pleasant side effect at that point.

aetolicus · January 9, 2019, 4:52pm

Hi, @Bitking

Unfortunately, I often get lost in your explanations. Am I getting it right that cells grids actually offer a solution to the combination part of the binding problem and not to the segregation part of it?

Thanks,
Dmitry

Bitking · January 9, 2019, 4:53pm

Yes, you do have this correct.
You are also correct that I need a good editor.

momiji · January 9, 2019, 10:43pm

Bitking can i clarify something with you.

The first and most important for me atm is how to treat the input that each mini-column / L2/3 cell would receive. In the spatial pooler implementation within a 2D input space (lets say MNIST) each mini column has a receptive field value (as it is an 2D array this is I guess a square around its centre). Within this receptive field (ie offsets from column index etc) a random number of input cells / array indices are chosen to be seen as connected, around 2% or however the original algorithm determines it.

If, as can be done already our spatial pooler outputs a 2D array of bits (i.e an 2D SDR), do you see each L2/3 cell / minicolumn as simply being a single / one to one mapping to the input SDR, or does each L2/3 cell also have a receptive field in which it can connect to multiple inputs.

This ignores the 0.5m dendrite ‘halo’ element of the hex grid activations.

I’m just curious to see what the best and most plausible approach to feeding in the input from the SP to a calvin layer / L2/3 layer would be.

I’ve been looking at: Project : Full-layer V1 using HTM insights to get an idea of the ‘dendritic extent’ as gimery puts it, but wasn’t sure how to handle the actual input part.

Cheers

Bitking · January 10, 2019, 6:45am

Repeating a key point: all of this is describing what is happening in the L2/3 layer. At this level all we are working with is patterns and NO predictive memory. The L2/3 is also the layer that talks with other maps though the output axons of the L2/3 cell bodies. Likewise, the sensory inputs and rising axons are projected upwards through all layer terminating in the dense mat of L1.

The projecting axons lateral branches do make connections with inhibitory inter-neurons not shown here.

I am certain that there are connections up and down the minicolumn between the layers but I am not ready to state exactly how these connections work. These connections are key to describing the relationship between pattern recognition and temporal prediction. Working out these rules would be key to understanding the training rules of predictive memory.

Now on to the parts relative to hex-grid theory …

If you look at my entry on the Project : Full-layer V1 using HTM insights post, #34 I put figures on the sizes of the various elements for minicolumn spacing.

Each cell in the mini-column has a few dendrites and each one is at least single SDR and maybe a few.

Referring to this picture from that post - each blue circle is a single mini-column. This is 100 or so cell bodies. Each cell body has it’s own dendrites - say 10 or so for a nice round number.

It turns out that the rising projecting axons are also spaced on this 30 µm spacing so you can assume that each blue circle also has a rising axon bundle.

The large black circle is the reach of the dendrites (+/- 250 µm, or 500 µm total) for the minicolumn in the center of the diagram. That gives the dendrites in each cell in each microcolum access to about 200 or so rising axon clusters. This can be thought of as a “receptive field” of the blue blue mini-columns and rising axons within this black circle for this minicolumn. This is all repeated for the next minicolumn. This is the repeating structure for all minicolumns in the cortex.

In this paper Horizontal Synaptic Connections in Monkey Prefrontal Cortex the lateral connections from the L2/3 cells are given as an average about about 500 µm.

The black beam in this picture is the long distance lateral connection between the two minicolumns so that the two minicolumn “receptive fields” connected by this link of the the hex grid covers the space with very little overlap and very little missed space.

When you factor in the fact that there are several long distance lateral connections emanating in all directions from each cell in the minicolumn you can see that the possibilities to form space covering hex grids is very large. Since these connections from any given cell do not fall on strict angles or length the individual cells can form hex grids with different angle, spacing and phasing.

So recapping, ever minicolumn has 100 cells that each have 10 dendrites, each potentially forming at least one SDR, possibly more. (at least 1000 SDRs per minicolumn)

Each Dendrite, if it went in a straight line from from the cell body passes at least 7 axon projection clusters and with branching probably many more. This means that the area around each minicolum is densely sampled with about 1000 branching dendrites which should end up sampling every rising axon cluster within reach of the dendrites.

The lateral connection link these minicolumns so that if they are responding to a learned pattern, even though it is sampled relatively sparsely, all the space in the resonating hex grid pattern is being sampled and bound together into a single larger unique pattern.

Here is a drawing relating the idealized concept to the messy biological bits.

Also see this post for more on the formation of larger patterns:

Bitking · January 10, 2019, 2:05pm

To directly answer this key point - the hex grid pattern IS the spatial pooler. I see this as a direct replacement for the spatial pooler in HTM theory.

The point of the pooler is to collect activations and sparsify them; the Hex-grid is a naturally sparse activation.