Two Types of Hierarchies?

As you just mentioned, the structure is rather malleable. Presumably to reflect the content.

The connections formed by learning - yes.
The underlying routing of fiber tracts - fixed by genetics

There is learning on all levels of structure, we are not insects. Higher-in-scale and lower-in-the-pipeline structures are more genetically influenced, but nothing is set in stone. Ok, that might be an exaggeration, to compensate for your exaggeration :). But even prenatal development is adaptive, which is a form of learning.

Anyway, effective structure is a composite of all levels, so learning in finer structures alters function and interaction of more coarse ones. For example, we all have two hemispheres, but the degree of lateralization generally increases during development, to the extent that differs between individuals. And lateralization, to whatever extent, is the top component of generalisation hierarchy.

So, large areas and their fiber tracts maybe mostly genetically determined, but hierarchical relationship between them is informed by learning. I assume that specific short-term pressures, or innate sensitivity to such pressures, would increase detail-oriented learning at the cost of broad generalizations.


I’m not sure if you are disagreeing with me or you just don’t like the way I say things.

I think we are in agreement on the gross structure of maps and fiber bundles. I think we agree that the function is strongly dictated by the learned connections at the terminus of these bundles. We agree that this learning is distributed in the maps from the primary sensory areas through to the lobe hubs or as they are often called - association regions.

In a prior post on the definition of intelligence I stated that the training absorbed combined with the gifts of genetics act to select the best actions in various settings. more intelligent people tend to pick what is thought to be better answers. I think we are in general agreement even though we are saying this in very different ways.

The original post asked about the hierarchical organization and my intent is that it is not what is normally thought of as a hierarchy but instead - a parallel data flow where the stream splits and rejoins to extract features of the stream. The representation maintains topology through this process. What is presented to the association region is a feature rich topological aligned stream that can be sampled for space-time features that add up to objects and spatial relationships. This is still in a distributed form that occupies some spatial extent of the association region and my be morphing over time.

In the case of the eye (processing of which consumes roughly 30% of the real estate of the cortex) the presented features strongly exploit the temporal nature of the processing to collect the topological organized stream of saccades to match up features from frame to frame for object recognition. Still not really what could be thought of as a spatial hierarchy although a case could be made for some kind of temporal hierarchy during recognition.

How does this differ from your understanding?

1 Like

Basically, you stress the negative: NOT all / most relationships are hierarchical, and then it’s NOT what “conventionally” is thought of as hierarchy.
I stress the positive: upstream areas contain at least as or more general, AKA stable, AKA invariant representations. (Edit: they are projected be as or more general according to prior experience, but may not turn out to be such in the future)

Which can only be discovered by expanding effective receptive field of these representations. It may expand across space or over time (the difference is often POV: one maps to another, as with your eye saccades), and this expansion is selective vs. contiguous in various ways.

But it’s not a question of hierarchy or not, just a shape and degree of deep vs. flat in that hierarchy.
Generalization in the cortex is like evolution in biology: it may not be consistent in form or direction, but nothing makes any sense without it.

Do you feel that this definition is informative?

My intent here was to stress individual variation in width vs. depth of hierarchical organization. I feel that lot of theoretical disagreements come down to mismatch between these parameters in the participants :).

This is what happens with you summarize an idea with a sentence that hits the key point - it is often weak and misses the supporting ideas. You know that I am not afraid to dump a wall of text to support an idea - just not every time I mention it in passing.

We have a long winded thread on intelligence vs consciousness and it is not topical to the original post - if you want to continue this idea I suggest that we switch to that thread but a short response is this:

I do.

In that thread there were several attempts to examine exactly what intelligence is. Most were a laundry list of traits that assumes that if you have these traits - you are intelligent. As you correctly stated above - intelligence is a continuum so you end up with a this multi-variate input function yielding a slope with no clear relationship between the factors in the definition and the result. It really misses defining the core of what intelligence is.

The definition I offer is simple and gives a general metric based on the concept that the purpose of having a nervous system is to select and perform some action based on perception. If you think about most measures of intelligence you can boil them down to measuring this exact thing. There may be internal factors that drive this selection of action but in the end - smart people understand what they are seeing and make better choices.

I am scared shitless of any philosophical discussion about “consciousness”.
It’s a purest form of intellectual porn.

I disagree, respectfully :).

I am sure you’ve seen my definition, but it didn’t strike a cord:
“Intelligence is a general cognitive ability, ultimately an ability to predict. That includes cognitive component of action: planning is technically self-prediction. And prediction is interactive projection of previously discovered patterns. This perspective is well established, pattern recognition is a core of any IQ test. But there is no general and constructive definition of pattern and recognition. Below, I define (quantify) similarity for the simplest inputs, then describe hierarchically recursive algorithm to search for patterns in incrementally complex inputs…”:

This actually does give a simple metric, which can be applied automatically on the lowest levels of processing. As I understand, your metric is a human judge observing some complex behaviour and applying a back-box load of implicit subjective criteria, case by case.

But it’s very to argue on this level of generalization…

1 Like

:100: :heart:

This fight is an uphill battle. Literally the majority of the machine learning and neuroscience communities still thing of the hierarchy in the classical way.

As you go up the hierarchy, each region still gets sensory input, but it represents less detail across a larger “field of view”. This means higher levels can identify very large objects before the lower levels and inform them so they can resolve their representations, allowing them to “learn the right object” as they inspect it.


But is the hierarchy merely a hierarchy of regions or is there a “scale hierarchy” at work too?


You can build a hierarchy from any point in cell space. Just look at one neuron’s connections (a map of maps). BOOM! hierarchy.

Looking at it this way, there are many hierarchies of “scale” as you say, depending on which layer you probe and which type of cell you hit. This hierarchy is just cell connectivity, and if you look around you in the world, you can observe hierarchy simply by picking some object and attempting to classify it’s parts and those parts’ parts etc. I think this is the type of hierarchy you are talking about.

I don’t think this is the same as the hierarchy neuroscientists talk about (although it absolutely is the type of hierarchy Deep Learning implements).


From my perspective, the main difference between classical hierarchy (used by DL implementations) and cortical hierarchy is the fact that abstraction is formed within the cortical column itself (not in the connections between cortical columns). This enables things like level skipping, reciprocal connections, or even (theoretically) depth-independent recursive hierarchies that you could never do in the classical tree-like structure. You still are building up abstractions the further you travel the pipeline, though, so it isn’t completely different from a traditional hierarchy from that perspective.


I think there is a general “scale hierarchy” in brain organization:
( cortical, sub-cortical,
( frontal / motor, posterior / sensory cortices
( lateral, medial cortices,
( dorsal, ventral streams,
( columns
( mini-columns
( neurons

Maybe missed some levels.

1 Like

I hate to beat a dead horse but I will take another whack at this one.
The “negative attitude” about hierarchy is a direct attack on this classic Christmas tree of concentration as one ascends the hierarchy:

A slightly modified version that still has most of the problems is multiple streams with sensor fusion somewhere at the top:

This hides the Christmas tree in each of the streams but implies that there is some sort of orderly progression from input to output, and that there is some high-level version that can be combined.

What I see after years of tracing out data paths is that everything is connected to everything and the connections are broad copies of each map to the next:

(Not the actual map, look at post #2 above for an example)

The WHAT and WHERE streams do eventually converge at the HC/EC complex, but everything in between is essentially a haystack of connections. Note that the processing at almost every map in the cortex is the convergence of two or more maps. I see that everything between the sensory cortex and the association regions is some sort of compare/contrast operation. After all these years I still have not been able to put my finger on an exact description of what the maps are doing with these inputs other than saying it is extracting features.


I guess that’s addressed to me. I never proposed “classical” hierarchy, it ignores temporal accumulation and discontinuities that result from pruning the search tree. I was talking about incremental generalization and composition, it has to be driven by search expansion in both space and time.

BTW, I think what and where converge in other places too. Specifically in inferior parietal, and indirectly in dlPFC


The classical description of hierarchy seems more useful if you are interested in things like anatomy and medicine, since it’s an approximation of connectivity. I think a lot of us, at least myself, get pretty annoyed by how neuroscience works because neuroscientists are more concerned with healthcare than AI.

I think the less detailed sensory input to non-primary regions is often like a different submodality. If your sensors describe a bunch of different things about the object, some of those things won’t be as spatially fine-grained as others. That can make it hard to tell whether a region is primary or higher order - does it receive direct sensory input because of level skipping, or because it processes a different aspect of the sense?

Some sensory submodalities arguably skip the first level of the hierarchy*. I think the targeted cortical region is still higher order if it doesn’t receive input from pathways directed up the hierarchy. At least, it plays the same role as a higher order region in the thousand brains theory. Maybe it’s better to talk about scale than hierarchy.

*E.g. a couple whisker pathways (VPM head and tail) only target whisker S1 in the septal columns and dysgranular zone (maybe specifically L4, I don’t remember), which are higher order if you define hierarchy by the CTC pathway and take the results from a few papers for granted. The thalamocortical axons spill over into the barrels but those might target distal dendrites of cells outside the barrels. As another example, some types of koniocellular cells in LGN might not target L3/4 of V1. I might be remembering wrong.


Let’s talk about both! Looking at a very large item that takes up your whole Field of View would depend heavily on higher levels of hierarchy, which get larger FOV from the sensory space. V1 is looking at very small FOV (like a straw).

Imagine an elephant standing a few paces in front of you. V1 might not have any columns that can identify the object, even with very high details of small areas. But higher levels will get bigger scale inputs, and be able to match objects it has seen at that scale. In this case, voting from the high levels of hierarchy can inform the lower levels on what the object is. But you can still recognize elephants in V1, they just have to be on the horizon, basically very small. Then V1 can inform your higher Visual levels what that hazy grey spot is.


But, in vision, at the sub-cortical tap level, the coarser sampling gives more weighting to the wider field and not as much to the expanded fovea.
The pre-attention before the scanning in the FEF and V1 serves to have a larger spatial extent in scene analysis.

The sub-cortex response by driving the eye to extract a stream of detailed features without as much concern about how apart they are. The balisitic distance of the scan fixations informs on the scaling of the object.

1 Like

The voting process is bi-directional, right? Each region basically creates a list of possible objects and they compare them to narrow down those lists further. If the connections between the regions are the same in both directions, that doesn’t seem like a hierarchy.
I recall displacements involving a pathway up the hierarchy. I don’t see the connection between that and direct sensory input to higher order regions (the idea of level skipping). I’m not saying that hierarchy isn’t part of HTM, just that it seems like each sensory pathway targets regions based on scale rather than always targeting the first several levels of the hierarchy. There could be regions at the first level for a broad aspect of the sensory input, on equal footing with a region for more precise aspects. I don’t see why it would have to only send the broader input to regions higher in the hierarchy.


Yes and no. As far as I have seen the connections are never one-2-one, it is always two or more regions projecting into the target.


I’d be careful of drawing conclusions from this, although it’s probably right for the most part since connections are usually imprecise. There are ~10 layers/sublayers, so 100 layer-layer connections, and they can direct in either direction or both, so that’s 300 ways for two regions to connect, plus maybe two trans-thalamic pathways (m and c) so 360. Even if a small subset of those connections are allowed depending on the relationship between the two regions, like maybe 10, most regions would connect to most others. The Christmas tree might be hiding somewhere in there, not that I think so.
Another reason to be cautious is that results can be wrong, which contributes to the sense of lack of organization I get when I research something for a while.