A Framework for Intelligence and Cortical Function Based on Grid Cells in the Neocortex

Thank you for taking the time to digest my wall of text. I had a hard time deciding on how much information was needed for the intended context.

I’m referring to the algorithm as described in HTM School with some of my own additions relating to inhibition and dendrite activity. I guess I’m a bit stuck in the nomenclature as it looked at least two years, or more, ago.

It would of course be best if the category stays stable but I’m not surprised that data of this sort can give unstable results. And yes, the training of the 2/3 layer is intended to give the same result as in the previous paper on object detection. As in, the first exposure activates as many possible object patterns as possible and for every new exposure, the state in layer 4 is biased by the state in layer 2/3. Layer 2/3 then uses this as a bias to further narrow down its own possibilities since the neurons in that layer have been trained to be activated by specific object patterns in layer 2/3.

My implementation follows my understanding of the previous paper on object detection. Maybe I’ve misunderstood how layer 2/3 is supposed to strengthen the internal connections between neurons in the same pattern. Either way, this training of neurons in layer 2/3 seems to help with the narrowing down of possible patterns with few exposures and my column performs worse if I remove this functionality.

Perhaps this is a result of me using a column with a magnitude fewer mini-columns than typically are being used in Numenta research and in Nupic. My reasoning when it comes to number of mini-columns is that if I get 100 mini-columns to perform well enough, having a magnitude more of them should result in dramatic improvements. I haven’t decided on what is enough but my gut feeling is that if I can reach a stable 50% with 100 mini-columns, stepping it up to the common 2048 mini-columns would make sense.

Further, getting a small network performing well enough to solve simple problems offers more opportunities when it comes to running the algorithm on very limited hardware.

Ah, ok. Then I’ll assume I’ve overlooked something. I’ll spend some more time with the paper.

This sounds a bit unlikely. If we look at the popular coffee mug example, many different coffee mugs will, taking subsampling and SDR attributes into account, appear very similar. For example, you can not sense colour with your fingertip so the same model of mug in a different colour will appear identical even though they in one sense are very different.

So, feeling a lip on the edge, a cylindrical form with a bottom and open top together with an ear starting somewhere close to the edge and terminating somewhere close to the bottom should make category detection very possible. It would, of course, be possible to get into more details with a finer sensor but I claim that moving from the category “mugs” to “mugs with texture on the outside” is a very small step.

I’d say that my results show that the ability to detect categories, even if not intended, seems to work on at least some level with a combination of location, sequence of sensory inputs and category biasing.

But, just to be clear, a network that has trained on mugs will of course not do well if you show it a cat or something from some other very different domain.

To me, this sounds like a description of what I’ve done. The sensory patch is small and projects to a small number of mini-columns. Sub-sampling removes even more of the information that is needed to properly separate a “1” from an “8” or a “4” from a “9”. Thus I let the sensor be exposed to overlapping patches (that are smaller than the training image) that offer separation of location for similar features and topological information that connects the features.

Going a bit further down the grid cell encoding rabbit hole: If the highest level of representation is at the autobiographical memory level and cognitively there is some sort of representation and relationships between objects there should be some sorts of basic operations.

This is exactly one of the issues I have been thinking about for many years. (my oldest notes on this run back over a decade)

One of the possibilities that keeps bubbling up as a strong candidate for internal representation is the tuple. That uses our internal spatial representation and system arranges our objects as (object)(relation)(object)

BTW: It’s nice that the rest of the world is starting to converge on the internal spatial representation that seemed most likely to me over the years!

With this long, self-congratulatory introduction - tonight I bumped into a very interesting paper that explores some of these same concepts:
https://www.ncbi.nlm.nih.gov/m/pubmed/30146305/
If you are the sort of person that uses a certain SH web page to view your papers you will be needing to use this DOI address:
doi: 10.1016/j.neuron.2018.07.047

Just posting an idea. This is highly likely not biologically possible.
We could make a grid sell encoder out of a capsule network. The routing mechanism works like a displacement layer. We could track where the values are ended up after routing, thus use it as the displacement.

Please read my post on Hex-grid cells. This is much simpler and more biologically plausible than using capsules.

1 Like

6 posts were merged into an existing topic: Intelligence vs Consciousness

This theory is so illuminating and beautiful, I really enjoy thinking about it and speculating further ideas. Many thanks to Numenta for sharing all these in an open and accessible way.

I have a question about “what” and “where” pathways. Let’s say I instruct another person/agent to manipulate an object and I already know the agent’s body space and behaviours well. So the task is to specify the movement in agent’s body space to get the desired location/state in the object’s space. Could it be possible that during this task, “where” region performs location computations on agent’s body space? If so, what could be the extent of spaces that “where” region compute locations on?

I will connect, this convergence is what will lead to the singularity…“We shall Ionize!i”

Based on your idea that there are “cortical grid cells” in L6 and “displacement cells” in L5, do you have any testable predictions to make about L5/L6 neurons?
@mrcslws @jhawkins

3 Likes

2 posts were split to a new topic: Why are humans special?

The paper makes several specific and novel proposals regarding the neocortex which means there are many ways the theory can be tested (both to falsify or support). In the posters we presented at the Society For Neuroscience conference this week we listed several testable hypotheses. Here is the poster about the new “frameworks” paper. It lists several testable predictions on the right side.

In practice it can be difficult to actually test these predictions. What is necessary, and what we do, is to sit down with experimentalists and carefully understand what their lab is capable of measuring and how that intersects the theory. This can take hours or even days just to design a potential experiment. For example, it isn’t known how capable rats are at distinguishing different objects via whisking (the active sense of moving whiskers). We predict that whisking should work on the same principles as vision and touch in humans, but we can’t ask the rat what it knows. We can’t even be certain that the whisking in the rat hasn’t evolved alternate strategies for operation. There have been recent advances in fMRI related to detecting grid cells in human neocortex. We list some of these in the same poster. fMRI might turn out to be a more fruitful experimental paradigm for testing the theory, but is limited in spatial and temporal resolution.

Bottom line is the theory makes many surprising predictions that should be testable, but it may take time to figure out how to actually test them.

5 Likes

How is a three dimensional cup mapped to a two dimensional grid cell array?

Have you seen this?

1 Like

I think this poster is even more relevant.

There will also be a paper soon about how 2D grid cell modules can track N-dimensional variables.

@rhyolight thanks. Do you have a Version of this poster with higher resolution?

Did you click on it for the higher resolution version?

1 Like

Aha, it is a problem with my iPhone… it works in my laptop as you mentioned…Thanks

2 Likes

This is a major step. Using the 3D case it explains why a grid cell ARRAY representation, it is a SDR, it really is HTM.

I would go further and say the point of grid cell arrays is not to make a homunculus of the world. It is not the one fully overlapping point that is useful, it is the SDR that is used.

I have a few questions on the article “A framework for Intelligence and Cortical Function Based on Grid Cells”. The questions are not on the basic idea yet, but on the grid cell theory behind it. You say that every learned environment is associated with a set of unique locations. So suppose you have two identical rooms, except one is colored blue and the other is colored green. You release a rat in one room, it learns its surroundings, then you release it in the other room, and it learns to get around in that room too. So it seems that the grid cells that are active at the left back corner of the blue room should be the same as the grid cells that are active at the left back corner of the green room. But it seems you are saying this is not true. If it is not, then why not? You also say that on entering a learned environment, grid cell modules anchor differently. Anchor means which grid cells are selected. Do you have a diagram that would illustrate this? Finally, in the example of the cup with the logo, why do two spaces exist - logo space and cup space? Are they both represented by the same modules? If they are, I would think there would be danger of overlap, unless you switch in time from one to the other and back.

Thanks in advance

1 Like

This is an area of great interest in grid cells studies.

Google “grid cells remapping” and find that as a critter enters and orients to a space the response of the ensemble of grid/place/border cells seems to be reshuffled.

I have yet to read a paper that is able to describe the principles of how it works - only that it does.

There are promising signs that someone will sort out how location is coded but these are still “early day.”

2 Likes