Thank you for providing your vision of the whole picture with supporting links.
Could you also elaborate on this statement?
I support this point of view, but could point to any supporting evidence?
Please look at links [7], [8], and [9] above for some basic information on grid cells.
I believe everybody is familiar with Mosersâ works here, but you mentioned it as a universal data structure format, however, to the best of my knowledge, itâs only a part of the navigation system. Do you have another vision of it?
I do.
Expand your view of navigation.
We know from patient HM that without the hippocampus you lose your ability to encode your navigation through experience. It makes some sense to think that the hippocampus samples the grid structure and rearranges or encodes the data that is proximal to the temporal lobe to be learned as your autobiographical experience.
The cortex is still forming the âglobal workspaceâ[1] but the mapping of navigation in this workspace is not being encoded and stamped into the temporal lobe.
[1] CONSCIOUSNESS AND COGNITIVE ACCESS NED BLOCK; See figure 4.
1 Like
This could be the where versus the what pathways of vision. If I am a fish and believe I am in a dangerous location and something appears to the right I know to fish fast to the left. On the other hand if I believe mates are near I choose to proceed in a courtly manner to the right.
For detailed âwhatâ saccading to center the region of interest makes sense. Orientation is covered by both Hawkinsâ theory and Hintonâs capsule networks. Hawkins being more biologically near and generalize-able.
Something when a thing is highly rotated I rotate my head to better see it, not something a mature adult does due to vanity.
I also appreciate your informative answer above. Can you please explain âdistributed gridsâ.? Thanks.
It looks you are talking about place cells (letâs generalize the term for direction and other functionally described types), not grid cells, which are located in the medial entorhinal cortex. Place cells donât have the grid structure.
There is good reason to believe that the place cells sample the grid cells with SDRs to form a pocket of response.
Sure, itâs quite clear, that the grid cells provide an important part of the input to activate some place cells.
I mean, grid cells are just one (even very important for real space navigation) of such inputs. Or, perhaps, it would be more clear to say, place cells can work without grid cells, but grid cells are useless without place cells.
Place cells by themselves donât have any grid or any other known to me structure. Thatâs why I was surprised to hear about universal data structure format in the brain.
This is an important question for both HTM and NNs. My understanding is that both do it in a disturbingly wasteful way. They both repeat detectors again and again all over the field and again and again at a different scale and again and again at a even coarser scale thousands of redundant detectors. Is there data that shows this is how biology works? pro side it is simple, con side it is wasteful.
I once heard a talk by someone from Columbia University on rate coding and how to use it to do transformations like scale and other things (placement, orientation?? not sure I remember this). If scale transformations are easy in rate coded spiking system it would be good to know that. One or a few detectors applied at many scale settings?
paper on vision in primates
Large-scale, high-resolution comparison of the core visual object recognition behavior of humans, monkeys, and state-of-the-art deep artificial neural networks
1 Like
Spin,
The entorhinal cortex is cortex. One of the takeaway lessons I have gotten from books like âOn Intelligenceâ is that cortex is cortex. The distributed grid representation is clearly spread over a very large chunk of the entorhinal cortex. Why should this computing pattern not be found elsewhere in the cortex?
The older allocortex in the hippocampus communicates with the neocortex and distills this representation into chunks like âplaces.â That strongly suggests that the place information is in the cortex distributed over this grid pattern to be sampled. Hence the claim of a distributed representation in the grid cells. SDRs are perfect to get a little taste of several different intermingled distributed representations from many sensory modalities and combine them into a meaning.
I have many reasons think that there are other codings like âplaceâ in the hippocampus but that they are not as easy to stumble into by monitoring free-roaming critters. It may be a real stretch to suggest that other parts of the brain use a grid pattern but consider this: we were able to sample grid cells because there was a strong relationship to something happening in the cage to correlate with. (See reference [7] above) I donât think I have seen anyone looking for this sort of property in other parts of the cortex.
I find distributed representation in grid patterns to be a delightful solution to the coding problem. Even if this turns out to be non-biological for most of the cortex it codes very well in neural networks. The ratslam people are doing very interesting things with this. In the H of HTM, we need to get from a point in the input pattern space and spread it out to interact with other parts of the pattern - a distributed representation. Humans have trouble visualizing this as we see in 2.5 D. (2D with stereo cues & color) Once we go from the highly topographic V1 cortex it gets hard to conceptualize how the data gets blended together. The V1 cells look like a topographic pattern because the retina feeds it that. Once we get into progressive visual maps it gets less clear what is happening to the representation. I can see how a sort of FFT transform could be spreading and combining different frequencies of sampled data across a distributed pattern in these maps. By the time we get to the association areas we have multiple streams of senses blended together. What is the lingua franca? Grids would work as well as anything here.
I agree that this is highly speculative and that it may be premature to assert that this is the ground truth but it does fit very well with many years of reading about this sort of thing - it just feels right. Win or lose I will be researching to attempt to learn if this is how the brain does it. I strongly suspect that the answer will be this or something very close.
1 Like
Bitking super interesting post. I expect it will take me years to understand. Thanks.
It took me years. On the other hand - nobody showed me what to look for.
Frankly, I canât find the purpose of using the periodical grit everywhere in the cortex. I was charmed by the hexagon magic of grid cells too, but after spending reasonable time in thinking about it, I didnât find any use for them outside the of creating a virtual space, what is super important for some modalities on the lowest levels of hierarchy, but how could it be useful in all other cases?
BTW, even the founder of the grid cell Edvard Moser, who continues his work in this field, quite often speaks about the importance of place cells for the memory mechanics, but not grid cells (he also sad the idea of FFT as the key mechanics for grid cells hadnât been confirmed).
I agree, that we are still on the early stage of collecting facts in this field and itâs very complicated and time-consuming to conduct research here, so some important information can be omitted. However, how the brain could use such grid as a general base for everything else? Perhaps, you could provide a simple imagining example for such mechanics for some nonspatial patterns to get the flavour of your vision?
1 Like
Consider the entorhinal cortex and what is being represented: a mixture of the vestibular system, visual representation, self-motion, postural feedback. Perhaps more. When you look at the repeating patterns where do you see any of that in the distributed dispersal that we choose to call a grid system?
I see that as the output of a common digestion system that ends up in a âcommon format.â The grid cells under discussion are a byproduct of explaining the coding that surrounds the hippocampus. Even though there are some very interesting theories to be best of my knowledge nobody really is sure of how that grid pattern is formed.
Looking at the dizzying maze of fiber tracts that connect the maps in the brain I have often wondered about the palimpsest of patterns that lay one on top of another in these many tracts. Consider the areas that receive the foveal vision as the eye darts over a scene, bit by bit - color, texture, stereo disparity, edges - all overlapping on the same maps. Within a few layers of maps that stew is mixed with the digested versions of body sensors (vibration, temperature, joint position and muscle stretch) to guide end-effector motion in space in the association areas.
The same general thing is going on in the auditory tracts.
Fat fiber bundles connect these diverse association areas and communicate some sort of meaningful information.
It gives me hope that if we can tease out how the wildly dissimilar information is combined in the grid maps that perhaps the same general process is working in everywhere in the brain.
You call grids a specialized navigation system - I see it as a possible general coding scheme that has been demonstrated to work in part of the brain. I will say it again - if the cortex is a general fabric that is uniform throughout then working out how part of it functions could apply to all of it.
Ask yourself - if you were going to take all of these wildly diverse sensory organs and apply a common process to turn it into a maximally dispersed (and compatible) coding to gain the best resistance to decay over time - what format would you expect it to end up as? For bonus points use a fabric of HTM style columns that do local processing to do this job.
1 Like
I really support everything youâve just sad, but I see generalized place cells in this role, not grid cells. Again, what could it be the reason to use the 2D grid for auditory modality and how can it be applied on abstract levels like the meaning of the speech?
Spin, because hearing is binocular. We need to place a sound on a 2D grid left/right and up/down. Exactly as we map vision onto a 2D grid.
To amplify this point: a next level of representation, speech sounds, can be arranged in a 2-d mapping arrangement.
http://www.animations.physics.unsw.edu.au/jw/voice.html