I haven’t really dived into this but it’s an area I find interesting and important and confusing.
I’ll focus on one particular example: As far as I understand, there are connections from primary visual cortex (V1) to the superior colliculus (SC), arranged topographically—i.e. the parts that analyze the same part of the visual field are wired together.
As you mention, Jeff has long said these connections are the cortex sending motor commands (superior colliculus is involved in moving the eyes / saccades, in addition to sensory processing). A new paper from S. Murray Sherman and W. Martin Usrey also says that these connections are motor commands. I don’t know who else thinks that, those are the only two places I’ve seen it.
Again I haven’t thought it too much, but right now I generally don’t like this theory. For one thing, my understanding is that V1 is not set up with the cortico-basal ganglia-thalamo-cortical loops that the brain uses for RL, and I normally think you need RL to learn motor control. For another thing, aren’t the frontal eye fields in charge of saccades?? (At least, in charge at the cortical level.) For yet another thing, it seems to me that “V1 cortical column #832” is not in a good position to know whether saccading to the corresponding part of the visual field is a good or bad idea. The decision of where and when to saccade needs to incorporate things like “what am I trying to do”, “what’s going on in general”, “what has high value-of-information”, etc.—information that I don’t think a particular V1 column would have.
The closest thing to motor control theory that kinda makes sense to me is a “Confusing things are happening here” message . More specifically, each V1 column ought to “know” if it’s the case that higher-level models keep issuing confident predictions about what’s gonna happen at that part of the visual field, and those predictions keep being falsified. So when that happens, it could send a “Confusing things are happening here” message to SC.
Those messages would not be exactly a motor command per se , but the SC could reasonably act on the information by saccading to the confusing area. So then the messages wind up being more-or-less a motor command in effect.
Then here’s one more alternative theory I was thinking about. There’s a thing where if there’s a sudden flashing light, we immediately saccade to it, and maybe do other orienting reactions like move our head and body (and maybe also release cortisol etc.). My impression is that it’s SC that decides that this reaction is appropriate, and that orchestrates it.
But if we expect the flashing light, we’ll be less likely to orient to it.
So maybe the V1 → SC axons are saying: “Hey SC, there’s about to be motion in this particular part of the visual field. So if you see something there, it’s fine, chill out, we don’t have to orient to it.”
I don’t know which of those three ideas (or something else entirely) is the real explanation, and again I haven’t looked into it too much, but I’m happy for an excuse to chat about it. 