Bitking, I think you should write something for neuroscience researchers. Youâve been accumulating knowledge for decades. For the sake of understanding how the brain works, neuroscience is sorely lacking summarization. I shouldnât have to spend a thousand hours just to understand the details of a single cortical layer, and I imagine the situation is similar up there at the level of maps and such. Heck, not just the details. Learning some essential things takes so so much sifting through papers. Itâs endless rabbit holes.
So whether or not your ideas end up working and being accepted, I think you have a lot to offer to neuroscience and indirectly, in the short or long term, AI. Even just outlining what sources exist and what they are on in a peer reviewed article would be very helpful. Summarize decades of reading and the contribution would seem to me as great as the findings by famous researchers. Youâd save hundreds or thousands of years of reading.
Planning to write a review article is how I deal with my lack of contribution to HTM theory, anyway.
Agreed. In the very first few lines of my hex-grids post I made that clear:
First - letâs be clear what I am and am not talking about.
This is about cells that work by forming regular hexagonal grid structures and are not necessarily the same thing as the âgrid cellsâ that are coding some spatial aspect of the surrounding environment.
These grid-forming cells that I will describe combine a sparse collection of mini-columns into larger hexagonal arrays. The cells that form a regular hexagonal array can offer a separate and highly useful tool in the neural network toolbox. They can act to take a local response from a mini-column responding to some locally sensed condition and join it to other local mini-columns that are also responding to what they see of their part of some locally sensed pattern. This is a form of lateral binding.
The Moser/grid cells use these hex-array forming cells to code the external sensed environment so we have a hex-grid signaling an external location (in a grid pattern) to the place cells in the hippocampus.
What in your opinion would be a good path to test out the hex-grid forming layer approach?
At least in a sense of seeing whether it fits within HTM, can be built on top of it as you mentioned in the original hex grid post. Iâm certainly wiling to give it a go programmatically, but whether it turns out to be the correct approach is another thing.
At any rate Im glad you pointed out this particular path of research, especially Calvins work Bitking!
The lowest hanging fruit is to take the code referenced in the columns paper and change the lateral connections.
The goal is to see groups of htm modules stepping through whatever learned patterns in parallel both in learning and replay.
Each htm mini-column would be learning different things but through the lateral connections these would be considered parts of a larger pattern. (Binding)
You donât need to change anything about the current HTM lateral connections, you can just run a process on top of it the activated minicolumns within a region.
Youâll have to ensure that there is a 2D topology enforced across all cortical columns that includes minicolumns. Each minicolumn must be able to efficiently calculate direct links to neighbors. And the calculation for that link needs to be programmable so you can tweak parameters for the CAN application.
For every active minicolumn, search for neighbors that are also active. At the end of the search you should have many grids of varying size. The larger the grid, the stronger the link with the SDR input.
This process could work within a cortical column, or across a population of cortical columns.
The all-to-all connections misses the interlocking sparsification and competition that arises from the hex-grid pattern proposal. This feature allows competing perceptions to attempt to match a larger spatial pattern; this is one of the important features of lateral binding of patterns.
Another area where a hex-grid forming layer could be applied is to improve the implementation for the Output Layer referenced in the columns paper (this is the area I have recently been exploring). The paper describes selecting a random SDR to represent each new object during a training phase which is required prior to performing object recognition. Hex grid formation could in theory be used to form these SDRs online from activity in the Input Layers, to eliminate the manual initial training phase. I believe this approach, if done correctly, should also capture object semantics as well (where similar objects have proportionally similar overlap in the representations in the Output Layer).
Consider this example of the possible set of input patterns on L1 that might be found on a given section of cortex:
We would like the trained mini-column to vote on what pattern they might be sensing and settle on one of these sets of output patterns on the projecting axons from L2/3:
This samples a tiny subset of these mini-columns in this area. The actual number of input-output pairs co-existing in this section of cortex could be a fantastically large number. Naturally, the number of activated grid elements is related for a given pattern to the size of the learned pattern; as time goes on the pattern could get larger as more âfence sittingâ elements are induced to learn the edge cases.
Note the important binding feature that allows mini-columns with very limited dendrite spans to work with very distant mini-columns to learn a very large continuous pattern. Sparsity is preserved at both the dendrite and the mini-column levels.
No - Calvin tiles work across large spans in a single map. In this case, I am focusing on the top of the hierarchy in a single sensory modality.
The H of HTM will work acoss maps. I have some very concrete proposals for that but this part of the theory (Calvin tiles) has to be elaborated before showing how the map-2-map parts work.
Spot on @Paul_Lamb! Exactly what I was thinking for some time. Glad I am not the only one.
My perspective is that hex grids seems like a more bio plausible alternative to temporal pooling that also has the potential to solve the binding problem / sensory fusion. It may also be a way to unify / merge the activation on distant cortical columns without explicit synapses between them. I think there is a lot to discover in this last sentence. I imagine I can only grasp a fraction of what this implies.
On the other hand, my main concern is the reduced capacity that results from hex grids because of the imposed local constraints.
I hope hex grids will have its say in my experiments soon.
