I sent a friend of mine Rotbart’s 2019 talk, a 30 minute general overview of the principles behind HTM. He sent me back some thoughts about how although structure is mostly uniform across the neocortex, there are distinct regions with specific characteristics.
I googled a little more regarding this, read some articles on how the primary visual cortex is located in the occipital lobe, complex language processing is mostly handled in the ventrolateral prefrontal cortex etc.
My neuroscience knowledge is very limited, so I’m making some rather broad assumptions here:
Unique regions handling certain functions mean that certain nerves are (genetically?) wired to input to those regions (eyes being largely connected to the occipital lobe)
Over the course of human life, nerve input to those segments influences neuron/synapse development according to 1) input data and 2) reward/consequence processing
Unique “styles” of combinations of input + ‘label’ will develop unique architecture / synapse connection structure within certain regions (visual processing neurons may look different from linguistic)
Therefore, if we’re able to analyze what makes the visual cortex’s neuron structure/connectivity/arrangement unique (compared to say, ventrolateral prefrontal language centers), we have some vague understanding of “what makes the brain good at processing a certain type of data”.
This is quite broad, still - “good” is in terms of ‘using the brain’s own biological machinery, hierarchical functions etc’, not necessarily good for existing HTM models.
So my question is: Are there any HTM systems/research initiatives focused on distinct regional variations within the neocortex? Or if we’re still far from that point, a good starting point for me would probably be to read some papers on neocortex regions and what makes the actual neurons unique.
I think the hard part would be translating those structural differences to an HTM model - variables like dropout, connectivity patterns and perhaps encoder tweaking?
This also made me think about running the same data through several HTM models with differently-tweaked parameters, not unlike a regular random forest model, though I doubt this is really a new idea.
Any thoughts regarding cortex regions and HTM architecture variance would be greatly welcome.
To my knowledge the focus has been mostly on variations between layers within the cortical macro column. In the talk he touches on this at the end with this diagram:
While Layer 4 handles raw sensory input, Layer 6 handles location of the sensor, and Layer 2/3 does pooling – which is shared between columns. This enables the columns to model sensory experience with the context of sensor location, and work collectively to identify what is being sensed (the cup) by all sensors (fingers).
That’s not to say that all cortical columns in cortex work exactly the same. I couldn’t tell you, though some of the more neuroscience experts maybe could. But overall I’d say the research has mostly focused on what is common throughout the cortex, since it is a lot and it appears to bring serious upside for applied AI.
There are visible differences between different parts of the cortex. This is how Brodmann was able to make his maps over 100 years ago.
Later work with other methods have confirmed that these divisions were correct.
So yes, the fiber tracts connecting the maps establish what is to be processed, and adaptations to the basic Internal wiring in those maps suit them to that processing.
I have good reason to believe that the basic arrangement is the same in all cortex but things like the ratio of lateral connections length and the size of inhibitory inter-neurons sculpt basic functions like Gabor filters and Calvin tiles.
So - how does this play with HTM? My hex-grid and Numenta’s Thousand Brains Theory are attempts to place the basic cortical column into a larger structure. At this larger scale the local features like connection lengths and inhibitory field sizes and types configure the larger functions each map performs.
Jeff argues in “On Intelligence” for “Common Cortical Algorithm”, i.e. that the whole neocortex is running a similar algorithm, albeit on different data. I basically agree with that; I have blog posts here and here, including references to further reading in favor of the hypothesis (“Rethinking Innateness” book) and against the hypothesis (a book chapter by Steven Pinker, an article by Gary Marcus). (If people have seen other good discussion on either side of the issue, please tell me!)
There are some anatomical region-to-region differences; I think they’re sorta analogous to hyperparameters on the cortical algorithm. Some of these differences have perfectly obvious explanations, like the fact that motor cortex has a prominent layer 5. Other differences I can only vaguely guess about, like what spindle (a.k.a. Von Economo) neurons are for (they are only in a couple parts of the neocortex). The majority of region-to-region differences I just don’t know about; I haven’t really studied it. I’m very interested in any attempts to not just list the differences between Brodmann areas but also explain them computationally—please tell me if anyone has seen any good discussion of that.
