What’s the difference between layers and levels in the context of the HTM theory? Furthermore, could you, please, explain what are regions, and how are they related to layers and levels?
Note: I roughly know the answer to these questions, but I think that a currently more qualified person than me may provide an exhaustive, well-written and clear answer (in a way that I couldn’t right now), which may be useful to the whole HTM community, especially to beginners. So, keep in mind that your answer should be helpful and understandable to newbies.
I will be the first to agree that naming in neuroscience could be better.
As I understand it, the sheet of the cortex is composed of what is generally recognized as six layers.
The outer layer (one furthest from the center of the brain) is called layer one
The bottom is generally called layer six. These are not hard and fast divisions as the whole concept of what makes up a layer is pretty loose at this time. There are some that consider the thalamus as the 7th layer.
This same sheet of the cortex has subtle variations that are discernable with different methods. Two of the oldest are inspection with low power magnification, or by noting what defects occur if some area is damaged - say by a war wound.
These older techniques have settled on the map offered by Brodmann:
these can be called maps, areas, or regions - all mean the same thing.
One of the newer techniques is to look with various 3D imaging methods and follow the connecting tracts - what is hooked to what.
This method has roughly doubled the number of identified maps/areas/regions.
The tool that has come out of this effort is the connectogram, see this list of maps at the bottom of this link:
It has been observed that in these map-to-map connections there are definite pathways. In these streams, there is a general scheme of progressing from the senses to the association region. Each of the sensory modalities seems to have this processing stream and this progression from senses to the association region is generally referred to as a hierarchy. There are similar streams in other places in the brain but that is more complicated than I want to tackle in an introductory post.
I want to add that these graphs that show a general stream make it look simple. This can be deceiving as in reality, the brain is very complex with a bewildering collection of skips and side connections in these streams. This paper should give a more realistic view of what a neuro-scientist is up against - see figure 1:
But wait - there’s more!
The progression of processing the main sensory streams seems to be split into two major paths - one is called the WHAT stream, and one is called the WHERE stream.
Do you want to dig deeper?
Here is some associated reading that should show these streams in action and give you a flavor for what is going on. You can skip these without guilt as they are more in-depth than most newbies are ready to digest.
Generally speaking, “levels of the hierarchy” are thought of as as the connections between regions. Note that most theorists that I have talked with agree that this isn’t a strict hierarchy. There are also horizontal connections between apparently separate hierarchies, feed-forward skipping levels, and so-on.
Here are some of Jeff’s comments on hierarchy, starting around 40:20. In fact that whole video might be worth a watch if you haven’t yet, since everything before that was discussing layers (so it should give you a sense for the difference between the two concepts)
To confuse things a little further, once you differentiate layers and hierarchical levels, there is I think an important point that may not be immediately obvious (it took me a while to wrap my head around initially).
I have frequently seen projects exploring hierarchies in classical HTM by sending the output of Temporal Memory from one region as input to Spatial Pooler of the next region, and so-on, forming the levels of a hierarchy. However, there seems to me to be a flaw with this approach, since the primary function of SP isn’t to increase abstraction, but rather to fix sparsity while preserving semantics. It doesn’t have a feature-binding property to it.
With that in mind, we must conclude that the actual logical boundary between any two hierarchical levels should in fact be located within the layers of a single region, rather than in the connections between regions. HTM theorizes that each region is internally forming stable object representations from its streaming input. This process is where, I believe, abstractions are being formed.
I am of course starting to move into the area of “Tangential Theories”, but I think this is still relevant to your original question (since it shows that there may not be such a clear physical line differentiating “levels of a hierarchy” from “layers”). In terms of SMI, this would be the process by which activity in the “Input Layer” forms representations in the “Output Layer” capturing the proper semantics of the object (i.e. semantically similar objects should have proportionally similar overlapping bits in their representations).
If we assume that this same process applies to both sequence memory and object recognition, then a simpler way to visualize the concept would be to take the activity from Temporal Memory and feed it into this process to form stable outputs that represent sequences (or parts of sequences). This was referred to in the past as Temporal Pooling. Visually, a 3-level hierarchy could be depicted like so (assuming SP between regions to fix sparsity):
Now obviously HTM theory currently has more than two layers involved in SMI, but these are still the important two conceptually for communicating my point. You can see that the transition from the input layer to the output layer within the same region is actually the logical boundary between hierarchical levels (not the transition from the output layer of one region to the input layer of another region). I think it is this architecture that supports the less traditional forms of hierarchy to be assembled.
@Bitking Let me ask you a few specific questions regarding your statements.
From my previous readings (and not going too much in detail), a biological cortical column (debatably) contains 6 layers. So, is the “sheet of the cortex” (as you name it) composed uniquely of these “vertical columns” of (mostly pyramidal) neurons (and connections between them)? If yes, I suppose this is the reason why HTM theorists are focusing on studying and modeling a single biological cortical column. Is this supposition correct?
So, Broadmann suggested a way to identify these slightly different “regions” of the cortex. The connectome and, in particular, the connectogram are, on the other hand, ways to visualize the connections between regions and, actually, you say, they allowed us to identify more cortical regions (I suppose, based on the connections?).
I suppose that these pathways are a sequence of connections between different “regions”. Is this what you (and other people) mean by “pathway”? By “definite pathways”, do you mean that some of these connections are, in a certain sense, not random, but they persist or seem to exist for a specific purpose?
By “streams”, I suppose you mean the “definite pathways”. Right? What do you more precisely mean by “scheme of progressing”? Do you mean the “way” (i.e. a “definitive pathway”?) the “information” flows from a sense up to the “corresponding” cortical region?
Does this imply that we have a “hierarchy” for each of our senses? If the answer is yes, does this mean that, in the context of the HTM, we have to model each of the senses using a different hierarchy? For example, if we wanted to solve a problem in computer vision using the HTM theory, I couldn’t possibly use the same hierarchy as one that has been used to solve a problem in speech recognition. Is this correct?
There is some confusion about what makes up a column or micro-column.
A vertical cluster of cell bodies runs vertically through the entire six layer stack and are spaced on 30 μm centers; these are frequently called a micro column and are actually visible in some parts of the cortex under magnification.
These column work in concert and are thought to group in columns of about 300 μm in diameter. I am not aware if there are physical features that can be discerned on inspection but scientists that measure the behavior of collections of cells do see this grouping in the activities of the cell populations.
The HTM algorithms enforce sparsity at the level that mimics this column density.
You are correct that HTM theory focuses on this at the basic unit of function repeated endlessly throughout the cortex.
@Paul_Lamb has done a nice job of explaining the basic distinction between the layers of the HTM model and relations to both the basic HTM theory and maps/regions above - I won’t try to duplicate this work as he has covered it.
I should note that the population of cells in the cortical sheet contains a bunch of other cells types that are not currently part of the HTM canon and will likely end up having to be addressed as the theory progresses.
My own work on Hex-grids uses the layer 2/3 to join columns into larger assemblies inside a single map.
Please see this post for more details:
These pathways for streams do seem to follow paths that originate in the areas more directly connected to the sensors that take in the outside world. These streams hop through several maps/regions and end up in the central hub of each lobe. The parietal lobe is tied to sight/sound/tactile senses and the hub that joins them is called the association region.
The frontal lobe is tied to the internal body sensor and also has a hierarchy of connected maps. These are not as well known as the parietal lobe but work on this continues.
The side lobes (the temporal lobes) are connected to other parts of the sub-cortical structures that seem to be more about values/emotions/memory and are also the store of your personal experience.
This collection of lobes are tied together with a collection of fiber tracts but the biggest bundles seem to be the ones that tie the hubs together. There is also a very special tract that seems to be both the driver for speech and very likely - the source of your conscious experience.
These connections of lobes have an overall plan of sensing and action that I describe in rough sketch here:
In neuroscience this larger pattern is often called the “global workspace” and there are some good papers on this:
If this interests you this is a good place to read more about it:
I would like to just say yes to your question but I think that is taking the cowards way out.
In general - each of the senses do follow a hierarchy from the raw sense to the association region. Evolution has also found that some fraction of this or that sense is useful for other purposes and has added other tracts connecting things that are not strictly part of this general path. One that comes to mind is the region/map that drives the eye position (frontal eye fields) has loops to the visual cortex letting it know that a position shift is about to occur and to adjust the processing stream to take this into account. There are other loops like this.
Yes.The results are fused in the association region. Also - don’t forget that these senses also are split up and processed in both the WHAT and WHERE streams.
Separate stream for sound? You may use a general copy of the visual stream but it will be a separate stream with its own special features for processing sound. Nobody said this would be easy.
Why the quibble about saying YES to your question? There is also a reverse stream that counter-flows TO the sensory maps. This originates somewhere in the fore-brain and passes somewhat through the sub-cortical structures - primarily through the thalamus. This stream is thought to be more of a command and control that helps direct the attention and activity of the sensory stream. Again - Nobody said this would be easy
There is also a profound and general scheme that loops from the motor drivers to the body (Somatosensory) sensors.
These two strips are on either side of the cleft (Central Sulcus) facing each other.
If you take the most direct connections from the motor drivers and skin/joint/body sensors there are layers or levels of connection as you move away from this central cleft.
These loops join equivalent levels of complexity and are thought to provide feedback control along with sensing. See the frontal eye field example above for some idea about how these things work.
BTW: I suspect that the experience of synaesthesia is due to a “stray” fiber tract connecting various sensory streams between the normal hierarchies.
