Affordance Competition Hypothesis

Are you sure you’re not just having cold feet?
:-p.

1 Like

No way! I think this is an interesting paper and I’m ready to discuss it. :smiley:

3 Likes

This is from a paper exploring the quasi philosophical questions of whether we should think of neuronal rates or phases as “coding”, at all.

Probably not, but I see why the philosophical question is, in fact, welcome.
What’s at stake for us, students of brains and wannabe modellers, is how we may get inspired in finding the pertinent questions, to get to insightful answers.
So I don’t see the true nature of the debate as ‘are brains representing such and such’? Rather, as an evaluation of the benefits of recentering our view on real (and sometimes initial) purposes of nerve cells.

We too often take the view that processing of sensory stimulus shall ultimately lead to a representation allowing us to “make sense of the world”… and we refine models so that they encode things we understand, and hopefully show stepping stones to those representations, imbuing the world with “meaning”. While, as P. Cisek would put it, what they do is often more ‘pragmatic’ than that. What sensory neurons activation was meant to carry originally was often the control of a quite straightforward behaviour.

Which spawns 2 thoughts:

  • A pragmatic representation is not necessarily readily amenable to understanding. As the author of the ‘Avida’ alife software said somewhere, optimized representations (such as the ones found by evo search, increasing a behaviour’s fitness) are as cryptic as compressed files… if they were not, ie if they were understandable, they wouldn’t be optimum.
  • Although… we “do” make sense of the world. And there shall exist something we could relate to somewhere. And some recognizable ‘encodings’ at times. Such as face cells and the like. But they’re likely high in the hierarchy or emerge from the concurrent activation of lots of more basic loops, having found some new purpose together, and possibly after having reached some critical “mass” (or we easily forget we’re so chauvinistic about human big brains and intelligence or consciousness at times).

That being said, those are my takes only. And I don’t understand in the slightest how plastic nets with STDP cells would finally decide to wire “usefully”, be it to specific motor nerve cells or to drive seemingly hard-fixed purpose neurons such as grid or orientation cells… But I guess some of you people are quite knowledgeable on these questions.

3 Likes

I don’t believe there is any reason to expect that some outside observer should be able to “decrypt” a brain’s activity. But does that imply there is nothing “encoded” in the first place? We could not possibly manipulate the world to the degree that we do if our brains didn’t encode vast amounts of information about it.

Of course, I get that Paul Cisek is referring to a far wider scope of brain architectures when he refers to so-called objective neural activity as “pretty rare”. But I think most of us on this forum are interested in mammal brains from the perspective of the cortex, where such activity seems (at least to me) as kind of its whole purpose.

2 Likes

Agree, I also found these concepts to be similar.

I also see these as the same thing. Behaviors and their effect on the relationship between components of a composite object can themselves be components of other composite objects.

Definitely. The idea of the cortex modeling the world in a way that is primed for action selection is pretty compelling.

4 Likes

Time to step up your expectations:

2 Likes

Our cortex is a huge part of the full “scope of [our] brain architecture”. So if you’re ready to accept something as pretty rare overall, I don’t think its rarity can be discarded as “not applying to cortex”.

TBH, I don’t know about the “pretty rare” thing myself. But I’m becoming more and more aware of the fact that cortex didn’t just appear out of the blue. If practical representations were indeed a thing, then even cortex deals with a whole bunch of those, and I’d bet still today.
Most of our cortical areas are from great apes, we have a great deal of those homologues still with primates, a fair amount probably homologous with mammalian LCO, and some allegedly before that. Start of our visual “streams” in particular are already wired to the pallium of lots of vertebrates. And even our hippocampus friend sitting “at the top”(?) has a very long history, before we knew how to breathe air.

4 Likes

If I understood the experiment you referenced, that required comparing “truth” with brain activity to build a sort of translation network. I’m not sure how well that would apply to something more abstract than decoding visual input, but would be (pleasantly?) surprised if such an “thought reading” interface turned out to be possible one day.

4 Likes

I wasn’t. I meant that given Paul Cisek’s perspective (as I understand it), I take the comment he made about rarity as likely not a reference specifically to our own brain architecture, but to brains “in general”.

1 Like

Baby steps.
The first aircraft were proof of concept; it took a while to get transatlantic and supersonic flight.
And built in kitchens and toilets.

2 Likes

Same question-mark :grin:

Great for locked-ins.

I could imagine a nightmare scenario where your response to questions is read involuntarily. On the other hand, there are certain repressive regimes that would welcome this tech with open arms.

3 Likes

Thanks for this.
I agree with the general sequence of events, but I think your explanation misses 2 things:

  • Parallel pathways with direct projections from the cortex to motor centers, along with the indirect ones via basal ganglia
  • The multiple loops necessary to converge to an action

Here is my take on this:

At any time, numerous L5 Pyramidal Tract (PT) cells of some cortical areas are broadcasting direct messages to motor centers & basal ganglia & other structures like high-order thalamic nuclei (those messages conveyed by axon collaterals are really identical). But those messages are not sufficient by themselves to activate motor centers that are under tonic inhibition by basal ganglia, except in rare cases when they are really very strong (like when we artificially highly stimulate the cortex).

If there are still incompatible competing affordances (=potential actions) in the cortex, the BG will maintain the corresponding motor centers under tonic inhibition, and order the thalamus to move the attentional spotlight towards other sensory flows that could help the cortical convergence towards the winner affordance.

After a few cortico-BG-thalamo-cortical loops, the BG accepts the winning affordance (which can now be called “action”) suggested by the cortex by stopping its inhibition on motor centers. The thalamus receives the exact same message, and then passes the information of the coming action to the cortex. The cortex can now actively predict the outcome in advanced and then compare it to the ground truth coming from the senses (active sensing). This is what we call perception.

During the next loops, the BG will automatically accept all subsequent actions suggested by the cortex that are part of the same sequence (the sequence of actions is stored in the cortex), except of course if an unexpected events necessitate to switch behavior. Thus, the cortex is free to unfold its sequence to motor centers. The cortical code of innate actions (like basic walking, swallowing, …) is very straightforward because all the complexity of movement unfolding is hardcoded in evolutionary ancient Center Pattern Generators (CPG). But for more complex actions, we need something to replace the hardcoded CPG. This is where the cerebellum comes into the picture with its primary role in unfolding (and learning to unfold) smooth sequences of motor commands.

To make it more understandable, I skipped the difference between action & behavior.
A behavior is a sequenced list of actions. The competition between behaviors is mostly executed in the BG and some of it has been delegated to the PFC in mammals.
An action is a list of a parallel commands specified in a given cortical area. But I prefer the term “action map”.

3 Likes

This is actually happening today! I’ll be online in about an hour.

2 Likes

The natives are restless!

It is over now!

Hello everyone. I just joined the forum and wanted to say how excited and flattered I am by all of the attention you’ve paid to my work. There is a lot to read through, here and in the other thread, so it will take me some time to absorb it and formulate some replies. But I will try to reply to the questions raised (at least to those where I think I know the answer) in the near future.

I should admit up-front that I’m not familiar with the HTM theory, so I can’t comment on the comparisons that some of you have made between it and my work. But I will read some of the papers and try to bring myself up to speed. It is always nice to find kindred spirits!

11 Likes

In this first posting, I’d like to address what I think is the most important contribution I can make to the discussions on this forum. It concerns the distinction, repeatedly made here and in the journal club video, between the “new” brain (mammalian neocortex) and the “old” brain (subcortical structures). I know that this is a popular distinction among psychologists and many neuroscientists, and it was the prevailing view when researchers like Vernon Mountcastle were doing their pioneering work. But with all due respect to those pioneers**, that view is incorrect.

The neocortex is not a recent structure that was added on top of an old “reptilian brain”, as previously believed. In fact, comparative and developmental work in the last few decades has thoroughly rejected the notion that the neocortex is a mammalian innovation, and some are now calling it “isocortex” to avoid the misleading implications of the prefix “neo”. Topologically speaking, the part of the brain that is the neocortex in mammals is extremely old, and might even have a counterpart in lamprey. For example, at the SFN conference in Chicago a few days ago, Sten Grillner’s group presented very strong evidence that within the layered lateral pallium of lamprey there is a retinotopically organized area that receives visual input via the thalamus (like V1 receiving from LGN), right next to two somatosensory regions (one spinal, one trigeminal), of which the former partially overlaps with a motor region projecting downstream (like the M1/S1 “amalgam” proposed for early mammals). They make a strong case that this is not a case of convergent evolution, but a homologous organization that was possessed by our common ancestor ~560 million years ago. So, with all due respect to Vernon Mountcastle (who is my academic “grandfather” and a hero), the view of neocortex that was prevalent in his day has since been disproven. Topologically, the neocortex is over half a billion years old and originally evolved to support very fundamental behavioral functions such as interactive control.

The discussions here of my affordance competition hypothesis have accepted the possibility that it applies to the “old brain” (and I’m glad to hear it!), but several of you feel that this shouldn’t apply to the “new brain”. I would claim that it does, if only because the distinction between old and new brains is a false distinction. The neocortex is not only old, but it is topologically inseparable from its connections with those other brain regions. In fact, instead of thinking of it as a separate structure with isolatable functions, another way to think of it is as part of three fundamental types of circuits: 1) the “dorsal pallial” portion of a set of parallel loops through the basal ganglia (just like the hippocampus is a “medial pallial” portion of a loop through the septal nuclei); 2) the telencephalic part of a set of parallel loops through the cerebellum; 3) the telencephalic part of a set of parallel loops through the external environment. These circuits are what define neocortical functions, and many people think otherwise only because of outdated ideas about evolution, and perhaps because cortical activity is easier to study with functional imaging than subcortical activity.

If one accepts the ancient origins of the neocortex (and the data is very strong on this point), then all discussion of old brain versus new brain should be abandoned. Furthermore, even if we want to focus on the mammalian neocortex, then we are led to consider it as fundamentally a sensorimotor structure. Take a look at the following diagram of a putative mammalian ancestor as proposed by Jon Kaas (2017):

Above, the neocortex is all the stuff above the rhinal sulcus, and note how much of it is concerned with sensorimotor control. This includes the areas labeled as RS, S1, S2, CS, V1, V2, as well as all of the medial shaded regions (cingulate and retrosplenial). Together, those make up what I’ve referred to as the topographic “dorsomedial neocortex” (following Finlay & Uchiyama), though I’d also presume to include MF. That leaves only a relatively thin strip of non-topographic “ventrolateral neocortex”, including OF, g, Aud, T, and perirhinal cortex. I’d claim that those latter regions are doing a lot of the action selection, including combining key stimuli with internal state information, and in the case of temporal cortex (T) eventually specialize in object recognition.

So in summary, the neocortex is an expanded part of a very old region that served sensorimotor control in early vertebrates, and most of it is still playing that role. When people like me record neural activity in frontoparietal cortex that’s what we find – a mixture of “sensory”, “motor”, and “cognitive” variables. But it’s not because the brain strangely combines these things. It’s because the combined thing is the thing, and has been all along.

Refs:

Kaas, J.H. (2017) “The evolution of mammalian brains from early mammals to present-day primates” (Chapter 3 of S. Watanabe et al, Evolution of the Brain, Cognition, and Emotion in Vertebrates: Springer Japan)

** I’d like to add that Mountcastle is my academic “grandfather” and one of the founders of my field, so when I say “respect”, I really mean it.

18 Likes

So wonderful to have you here!

If you want a quick introduction, I can’t suggest enough to watch this presentation by @jhawkins. It’s a great overview and the fastest and easiest way to get up to speed.

After that, the most important papers are probably these four (although there are many more excellent papers).

Why neurons have thousands or synapses - A theory of sequence memory in the neocortex

A Theory of How Columns in the Neocortex Enable Learning the Structure of the World

Locations in the Neocortex: A Theory of Sensorimotor Object Recognition Using Cortical Grid Cells

A Framework for Intelligence and Cortical Function Based on Grid Cells in the Neocortex

6 Likes

HI Paul,
Thank you for contributing to the forum. I think our views on the evolution and phylogeny of the neocortex are more similar than you think. It is clear to us that the neocortex, or if you prefer isocortex, is not a “new” organ in that is an evolved version of previous and older structures. But it isn’t exactly the same either. There are several laminar structures such as the hippocampus, entorhinal cortex, and retrosplenial cortex that have some commonality with neocortex but also vary in large and noticeable ways. What is remarkable about the neocortex, in our opinion, is how quickly it became so large. Mountcastle proposed that the neocortex got large by mostly replicating a common structure. The rapid expanse of the neocortex is strong evidence he was right about that, regardless of how the common structure evolved or how similar it is to much older structures in the brain. It is the rapid expansion of the neocortex within the past several hundred thousand years which is why we feel justified in calling it “new”.

We believe that the neocortical circuitry is a refined and repackaged version of older laminar structures. In our 2019 paper, “A Framework for Intelligence and Cortical Function Based on Grid Cells in the Neocortex”, we argue that the circuitry in a small part of the neocortex, a cortical column if you will, is a repackaging of grid cells and place cells. We argued that columns in the neocortex learn models of objects in the word using the same basic cellular mechanisms as the hippocampal complex learns models of environments. That is, each cortical column has equivalents to grid cells, place cells, and head direction cells.

Any animal that exhibits goal oriented behavior is probably using the same basic modeling circuits someplace in the brain. Under this view, learning models of the world using metric reference frames (such as grid cells) evolved a very long time ago and that something akin to grid cells exists in many brain regions in many non-mammalian species.

If you have a chance to read our 2019 Frameworks paper I would appreciate hearing your thoughts as it relates to your work.
Jeff

10 Likes