I’m sure this will be a little controversial, but I’m not at all convinced that all cortical columns are actually generating motor commands. Jeff has yet to convince me on that. In motor regions perhaps, but not in all cortical columns. They may be projecting to motor-regulating subcortical regions, but that doesn’t mean they’re actually generating motor commands.
I have my own hypothesis about motor control in HTM, and the role I believe the Basal Ganglia plays relative to HTM theory. That thread is pretty old at this point and could maybe use some revisions, but I think all the main parts of my hypothesis are explained in the thread, even if they’re a bit unorganized. I should maybe redo my explanation of it sometime, and how I think it fits with Numenta’s current ideas. Some of the ideas I had there were kind of a primitive form of the Thousand Brains Model (though this was from before Numenta started discussing that theory, and my thread doesn’t explain it well). I’m not yet sure how grid/displacement cells play with it, but that definitely seems like a very interesting subject.
A summary of my hypothesis is this:
- The striatum is doing a similar temporal memory algorithm to the cortex, however with a reinforcement-learning twist.
- The dopaminergic receptors on the striatal cells use inhibition and disinhibition effects to group striatal cells into 3 subpopulations; those biased to respond to high-dopamine situations, those biased toward low-dopamine situations, and those with no significant bias one way or the other. These can be thought of as learning patterns/sequences that result in good/bad/neutral outcomes.
- The extra circuitry in the basal ganglia (globus pallidus, etc.) seems perfect for detecting when “good” patterns are detected, and no “bad” patterns are detected (e.g, the action the cortex is proposing is more likely to result in positive outcomes than negative ones). When this occurs, signals are relayed back to the cortex.
- The basal ganglia and cortex form many loops, that seem to have relatively small receptive fields. The cortex projects to the striatum, which projects to other areas of the basal ganglia, and those eventually project back to the same area of the cortex, likely within the same cortical column. These loops only feed back to the frontal lobe however. The frontal lobe is structured as a hierarchy, and almost all of it forms these loops with the BG, forming a hierarchy of reinforcement learners.
- The feedback from the BG to the cortex appears to be selectively disinhibiting/exciting cortical columns that are producing “good” actions, and possibly inhibiting those creating “bad” actions.
- Motor commands seem mostly driven by population coding. If the cortex is left to learn on its own, it will tend to assign neurons to patterns randomly, which doesn’t lend itself to population coding well. The BG in this model would be an extra force biasing neurons toward representing patterns strictly associated with useful commands. If the BG were to stop functioning properly, this force would then disappear, and neurons would fall into a new local minima and begin organizing themselves randomly again. It’s conceivable that this would result in erratic motor behavior, which is exactly what is seen in degenerative diseases of the BG (Parkinsons, Huntingtons, etc.).
A few things to note that probably are not mentioned or explained in depth in the original thread:
- Every part of the cortex projects to the striatum (which is where I think Jeff is getting the idea that every part of the cortex has motor output), but only the frontal lobe gets feedback from it. I think this lines up well with the Thousand Brains Model; these other areas aren’t generating motor commands, but are providing context. If the motor cortex suggests reaching your hand out in front of you, that may be a good idea if there’s a friendly cat there, but not a good idea if there’s a hot stove. In that case, non-motor information may be necessary for the BG to make informed judgements on the outcomes of actions. This doesn’t require motor commands from these regions; just information on objects, locations, etc. Perhaps there’s something interesting with displacement cells going on here.
- Also related to the Thousand Brains Model, the frontal lobe seems to project back to many sensory regions of the cortex. Even if the sensory regions aren’t creating their own motor commands, something with the appearance of motor commands could perhaps emerge from the interplay between these regions. I don’t have any concrete or well-developed ideas on this yet, however.
I don’t see any way in which the cortex could form any real motor commands on its own. Motor commands would imply that there’s some form of intention or goal that the cortex is attempting to achieve. Without reinforcement learning, I struggle to see what mechanism in the cortex could be driving that.
Like I said though, something interesting might be going on with displacement cells. I’m inclined to say that this is unlikely to be motor commands, but perhaps some kind of map of where features are relative to each other? That seems like it would be straightforward for the cortex to generate while being very useful for the striatum in judging the effectiveness of commands, as well as in path integration.