Yes - I am slowly working through the explanations and drawing for that portion. It is somewhat more complicated and that is taking some time.
I am also working on the lizard brain from the tip of the toe through to the forebrain with details of the lizard brain command structures.
I am also working on the interaction between the lizard brain and the cortex. Lots of drawings needed here. This also gets heavily into the counterflow of information back down towards the sensory streams.
I am also working on the capture and formation of semantic content from the sensory stream through to the temporal lobe.
Each of these is about as detailed as what I have going here for the grids and I can see how they all fit together.
Only so many hours in a day and I still have a demanding day job.
That’s great. I would like to know your current thoughts in detail. I have some theory of how the grid patterns might be working at different scales and layers to form representations about location at different abstract “space grids” and then later combining into groups of locations or multiple space-grids. This is like combined manifolds you mentioned in another thread. I think your explanation of combinations of grids might come handy in the theory.
There are nice ideas in the writing. The most interesting for me was the implications of local inhibition at higher levels of abstraction. New relations emerge. We mostly go with global inhibition in HTM layers at the moment which may cost us some emerging relations when the hierarchy is in place. It was probably hard to structure but the general flow may be represented better, I was kind of lost on the thought process at times. On a different note, it would be great if you could find a way to test some of the ideas here at small scales to be certain about their implications before moving forward at various stages to make the best of your time. How you would do that, I do not really now. Thanks for the writeup.
Man oh man, this post is great! I’ve wanted to get a quick getup with the theory between HTM, the layers and all that jazz. And this post seem to be the perfect starting point. The vulgarized explanations with all the graphics are godsend.
You’ve truly outdone yourself and shown what is high quality posting Bitking!
Check out Matt Taylor’s superb Grid cells presentation in the HTM school!
Matt points out that thinking of grid cells as maps of a room vastly underestimates the general potential of mentally mapping any arbitrary 2D space.
Matt demonstrates the relationship between SDR representation and grid coding. I outline multiple methods of using HTM to form grids in this post. There are some pieces to put together but this combination represents a potential tool to code a reasonably intelligent agent. So much to do to tie this all together.
He also introduces the concept that grid cells might be used in the neocortex to represent all objects in the brain, not just locations relative to an agent’s body. The papers below show how general mental cognition may well rest on this ability to mentally represent and manipulate space.
Here are the links referenced in the video comments:
Mapping of a non-spatial dimension by the hippocampal-entorhinal circuit (Dmitriy Aronov, Rhino Nevers, David W. Tank):
Organizing conceptual knowledge in humans with a gridlike code (Alexandra O. Constantinescu, Jill X. O’Reilly, Timothy E. J. Behrens):
Organizing Conceptual Knowledge in Humans with a Grid-like Code
Alexandra O. Constantinescu1, Jill X. O’Reilly, Timothy E. J. Behrens It has been hypothesized that the brain organizes concepts into a mental map, allowing conceptual relationships to be navigated in a similar fashion to space. Grid cells use a hexagonally-symmetric code to organize spatial representations and are the likely source of a precise hexagonal symmetry in the functional magnetic resonance imaging signal. Humans navigating conceptual two-dimensional knowledge showed the same hexagonal signal in a strikingly similar set of brain regions to those activated during spatial navigation. This grid-like signal is consistent across sessions acquired hours and more than a week apart. Our findings suggest that global relational codes may be used to organize non-spatial conceptual representations and that these codes may have hexagonal grid-like pattern when conceptual knowledge is laid out in two continuous dimensions. http://discovery.ucl.ac.uk/10043572/1/0.merged_final.pdf
Sorry for my question, which may simply come too early for my current understanding. I haven’t digested all this yet. But… well, grid cells and their power for computing a position in the environment’s topology, as presented in the paper “Connecting multiple spatial scales to decode the population activity of grid cells”, versus a cortical grid-like topology seem like two entirely different matters to me. Granted, I myself had random thoughts which erred towards that same direction, but what makes you so confident that there would be some actual correlation ?
(Still haven’t read Calvin. I’d like to make clear that I’m not refusing the idea of grids in the brain topology here - although I may raise same concerns as Paul for a usage with current HTM model. I’m simply curious about the correlation you obviously sense between 2014-nobel-grid-stuff and grids within cortex itself)
Different levels of representation
Local topology forms grids of various spacing.
It’s baked in.
When these different sections do their grid forming thing it favors a particular spacing due to its genetic wiring in that area. When the efferent fiber tracts from different areas converge on some distant map there are samples of multiple scaling represented in the combined population of activation. This behavior is at the H level of HTM theory - topology matters greatly at this level.
I strongly suspect that all grid-forming areas are built up from multiple semanticly related sources. I may be putting the cart before the horse here - the fiber tracts that combine in an area define a semantic meaning that is the intersection of the semantics of the source areas - which may seem in-turn be the semantic combination of its sources.
I think of the purpose of the grid topology is to form a united excitation pattern where the grid members are all recognizing the same thing at the same time. This forms and projects this knowledge to the next map: it’s saying that “I saw this thing at this place in my sensory space.” Instead of a few random bits, you have a (sparse) spatially widely distributed pattern; way bigger than the dendrite reach of any single pyramidal cell.
This locally addresses what (for some) is THE holy grail problem in neural representation: Binding.
I believe that inter-map reciprocal projections extend this pattern formation to global binding.
Very well said. I think this notion of referring to the grid mapping with reference to the cortices own natural processing space has its own significance.
The idea of forming semantically spaced grids with respect to the sensory information and their semantics is different and more complex in my mind. It calls for its own additional pattern processing.
The symbols should be at the same level of representation; I find it difficult to imagine that different levels of abstraction could exist in the same map. I see it more as the map in that area could represent more symbols at the same time. You can keep more than one “chunk” of information in your mind at the same time - this could be the mechanism that makes that possible.
But why do you think how you feel about something is a different level of abstraction…I think it’s just a different kind of abstraction. Why couldn’t emotional data manufactured in the limbic system be fed into the neocortex as an input stream just like vision. HTM Theory says it doesn’t matter what the data stream is so emotional patterns could be stored as memories or feature data in exactly the same way as any other input stream.
I was just thinking about it and it seems to me that it could be that emotions don’t really exist? (Just as I believe “thinking” doesn’t either…).
If we’re really honest about it, what passes as thinking usually is going over the “known” over and over again until a new insight “shows up”. It’s not like we do anything physical which directly results in a thought - they just show up.
I think emotions are even more elusive because they are always in the domain of an observer (even if it’s ourselves), as a characterization of physical sensations we have and behavior that is the result of those physical sensation’s exhibiting force on us in a particular behavioral direction.
So if emotions are labels we put on behavior resulting from bio-chemical changes which we interpret as emotions - then my question is (assuming what I’m saying is true), why would they (emotions) require any cognitive processes?
Sure, they require cognition in order to “observe” them - but produce them or experience them?
I make this claim because I have been looking at the topology of the connecting fiber bundles.
A map only knows what is projected to it. For example: V1 is receiving bundles of fibers from the eye so I don’t expect to see it processing auditory information.
The functions of various areas of the limbic system have been mapped, as have the connections between the limbic system and the cortex. I don’t recall seeing connections between the recognized cortical hubs where grids are formed and the parts of the limbic system thought to form emotional coloring.
These limbic projections target the lower frontal lobe and the temporal pole. These are the areas where, respectively, high-level planning and evaluation of your episodic memories reside.
One thing I would also like to comment on is what appears to be a “collapsing” of scope or dismissal of scale or level at which we are working?
What I mean is this…
From a single input bit to the internal verbalization of a concept or idea probably results from maybe hundreds or thousands of “layers” of SDRs provoking pathways resulting in more SDRs until distinctions are refined and combined with many senses; later resulting in guttural sounds; then words; then concepts and ideas?
We always talk about cortical processes as if we’re at the very “top” of the assembly and propose analogies to very complex concepts and abstractions when those things are probably the result of thousands of combinatorial processes occurring before we arrive at the complex concepts we ascribe to the SDRs we’re speaking of…??
For example, we talk about the conceptualization of a “Cup” and it’s constituent parts and the SDRs which contribute through their combination with other feature SDRs to their formulation as a formal “Cup” concept…
…when really we are at a very very very preliminary “scope” at first and probably have to deal with things such as what it means to “touch” something, detecting it’s smoothness and temperature etc. We probably won’t come to anything that can be distinguishable as a “thing” conceptually until HTM Theory is able to formulate thousands of layers of combinations to even arrive at a “word” for something? Maybe?
Edit: My personal opinion is that what we’re actually dealing with is a repeatable cognitive heuristic which will eventually result in an emergent concept after many thousands of prior combinations even wayyyyy before distinctions get characterized in language?
We use analogy to describe complex concepts so that we can “envision” the flow and assembly of units of cognition - but it seems that we forget that we’re not yet at that “scope” yet when we’re actually at a very basic level at the moment?
I just thought it might be useful to point that out?
You are making this too hard. The “hundred step” principle puts a very hard upper limit on how much is actually being done from perception to response. Everything from the formation of a “global workspace” to the initiation of action has to be a fairly short process.
But actually I’m not making any statement as to how complex the process is - just that there may be some benefit in being more “attentive” to the rigor with which we talk about the cognitive processes we’re currently working on? Basically that there is a looooong way to go before we even deal with the distinction of “things” because that requires the integration of language - and to me we’re at a much more “preliminary” stage in our development, maybe?
But anyway, I don’t mean to obfuscate the conversation?
From my reading, I find that the neuroscience community knows some things in a continuum from excruciating detail to a “vague fuzzy feeling.”
From fMRI studies, we can tell things like a given word or activity activates a particular region or regions.
From tract studies, we can tell to a high degree what regions are connected to what regions and the relative density of those connections.
Developmental studies have shown how the connections form in the developing animal.
Lesion studies have done much to catalog and localize function. Man (via war injuries) has been conducting highly detailed focal injuries experiments to extend this lesion damage knowledge.
From microscopic studies, we have some very good ideas on where the connections are made in the cortical sheet. Perhaps 20% of the neurons in this menagerie have some working theory on what function they provide - many that have strong support via “in vivo” studies. These in vivo studies have received an amazingly powerful tool in light activation and/or light emission genetic manipulation.
Phycological studies have done much to give some good “black box” description of what tasks are being performed by the brain. Other studies have done much to elaborate the order that the brain learns and exhibits these behaviors.
Interspecies comparisons add detail on what functions go with what structure and configuration of the structures.
So - how much is “known” depends to a great degree on how much you are willing to dig and integrate. Whether something is a “preliminary” stage or some more advanced stage may depend on your personal journey. Until someone puts it all together “we” may not know how long that journey is.
It never fails to amaze me that once I get some question in my mind regarding neurobiology I look and - lo an behold - someone has been researching it. There it is - laid out in research papers replete with tables, graphs, measurements, and references.
I think that we are in the same place as chemistry was in 1869 when Russian chemist Dimitri Mendeleev started the development of the periodic table. Once we have the “periodic table of the brain” everything may fall into a framework that makes everything make sense. I predict that we will discover that we really did know the answers - we just did not know how to fit the pieces together.