If I understood the experiment you referenced, that required comparing “truth” with brain activity to build a sort of translation network. I’m not sure how well that would apply to something more abstract than decoding visual input, but would be (pleasantly?) surprised if such an “thought reading” interface turned out to be possible one day.
4 Likes
I wasn’t. I meant that given Paul Cisek’s perspective (as I understand it), I take the comment he made about rarity as likely not a reference specifically to our own brain architecture, but to brains “in general”.
1 Like
Baby steps.
The first aircraft were proof of concept; it took a while to get transatlantic and supersonic flight.
And built in kitchens and toilets.
2 Likes
Same question-mark 
Great for locked-ins.
I could imagine a nightmare scenario where your response to questions is read involuntarily. On the other hand, there are certain repressive regimes that would welcome this tech with open arms.
3 Likes
Thanks for this.
I agree with the general sequence of events, but I think your explanation misses 2 things:
- Parallel pathways with direct projections from the cortex to motor centers, along with the indirect ones via basal ganglia
- The multiple loops necessary to converge to an action
Here is my take on this:
At any time, numerous L5 Pyramidal Tract (PT) cells of some cortical areas are broadcasting direct messages to motor centers & basal ganglia & other structures like high-order thalamic nuclei (those messages conveyed by axon collaterals are really identical). But those messages are not sufficient by themselves to activate motor centers that are under tonic inhibition by basal ganglia, except in rare cases when they are really very strong (like when we artificially highly stimulate the cortex).
If there are still incompatible competing affordances (=potential actions) in the cortex, the BG will maintain the corresponding motor centers under tonic inhibition, and order the thalamus to move the attentional spotlight towards other sensory flows that could help the cortical convergence towards the winner affordance.
After a few cortico-BG-thalamo-cortical loops, the BG accepts the winning affordance (which can now be called “action”) suggested by the cortex by stopping its inhibition on motor centers. The thalamus receives the exact same message, and then passes the information of the coming action to the cortex. The cortex can now actively predict the outcome in advanced and then compare it to the ground truth coming from the senses (active sensing). This is what we call perception.
During the next loops, the BG will automatically accept all subsequent actions suggested by the cortex that are part of the same sequence (the sequence of actions is stored in the cortex), except of course if an unexpected events necessitate to switch behavior. Thus, the cortex is free to unfold its sequence to motor centers. The cortical code of innate actions (like basic walking, swallowing, …) is very straightforward because all the complexity of movement unfolding is hardcoded in evolutionary ancient Center Pattern Generators (CPG). But for more complex actions, we need something to replace the hardcoded CPG. This is where the cerebellum comes into the picture with its primary role in unfolding (and learning to unfold) smooth sequences of motor commands.
To make it more understandable, I skipped the difference between action & behavior.
A behavior is a sequenced list of actions. The competition between behaviors is mostly executed in the BG and some of it has been delegated to the PFC in mammals.
An action is a list of a parallel commands specified in a given cortical area. But I prefer the term “action map”.
3 Likes
This is actually happening today! I’ll be online in about an hour.
2 Likes
The natives are restless!
It is over now!
Hello everyone. I just joined the forum and wanted to say how excited and flattered I am by all of the attention you’ve paid to my work. There is a lot to read through, here and in the other thread, so it will take me some time to absorb it and formulate some replies. But I will try to reply to the questions raised (at least to those where I think I know the answer) in the near future.
I should admit up-front that I’m not familiar with the HTM theory, so I can’t comment on the comparisons that some of you have made between it and my work. But I will read some of the papers and try to bring myself up to speed. It is always nice to find kindred spirits!
11 Likes
In this first posting, I’d like to address what I think is the most important contribution I can make to the discussions on this forum. It concerns the distinction, repeatedly made here and in the journal club video, between the “new” brain (mammalian neocortex) and the “old” brain (subcortical structures). I know that this is a popular distinction among psychologists and many neuroscientists, and it was the prevailing view when researchers like Vernon Mountcastle were doing their pioneering work. But with all due respect to those pioneers**, that view is incorrect.
The neocortex is not a recent structure that was added on top of an old “reptilian brain”, as previously believed. In fact, comparative and developmental work in the last few decades has thoroughly rejected the notion that the neocortex is a mammalian innovation, and some are now calling it “isocortex” to avoid the misleading implications of the prefix “neo”. Topologically speaking, the part of the brain that is the neocortex in mammals is extremely old, and might even have a counterpart in lamprey. For example, at the SFN conference in Chicago a few days ago, Sten Grillner’s group presented very strong evidence that within the layered lateral pallium of lamprey there is a retinotopically organized area that receives visual input via the thalamus (like V1 receiving from LGN), right next to two somatosensory regions (one spinal, one trigeminal), of which the former partially overlaps with a motor region projecting downstream (like the M1/S1 “amalgam” proposed for early mammals). They make a strong case that this is not a case of convergent evolution, but a homologous organization that was possessed by our common ancestor ~560 million years ago. So, with all due respect to Vernon Mountcastle (who is my academic “grandfather” and a hero), the view of neocortex that was prevalent in his day has since been disproven. Topologically, the neocortex is over half a billion years old and originally evolved to support very fundamental behavioral functions such as interactive control.
The discussions here of my affordance competition hypothesis have accepted the possibility that it applies to the “old brain” (and I’m glad to hear it!), but several of you feel that this shouldn’t apply to the “new brain”. I would claim that it does, if only because the distinction between old and new brains is a false distinction. The neocortex is not only old, but it is topologically inseparable from its connections with those other brain regions. In fact, instead of thinking of it as a separate structure with isolatable functions, another way to think of it is as part of three fundamental types of circuits: 1) the “dorsal pallial” portion of a set of parallel loops through the basal ganglia (just like the hippocampus is a “medial pallial” portion of a loop through the septal nuclei); 2) the telencephalic part of a set of parallel loops through the cerebellum; 3) the telencephalic part of a set of parallel loops through the external environment. These circuits are what define neocortical functions, and many people think otherwise only because of outdated ideas about evolution, and perhaps because cortical activity is easier to study with functional imaging than subcortical activity.
If one accepts the ancient origins of the neocortex (and the data is very strong on this point), then all discussion of old brain versus new brain should be abandoned. Furthermore, even if we want to focus on the mammalian neocortex, then we are led to consider it as fundamentally a sensorimotor structure. Take a look at the following diagram of a putative mammalian ancestor as proposed by Jon Kaas (2017):
Above, the neocortex is all the stuff above the rhinal sulcus, and note how much of it is concerned with sensorimotor control. This includes the areas labeled as RS, S1, S2, CS, V1, V2, as well as all of the medial shaded regions (cingulate and retrosplenial). Together, those make up what I’ve referred to as the topographic “dorsomedial neocortex” (following Finlay & Uchiyama), though I’d also presume to include MF. That leaves only a relatively thin strip of non-topographic “ventrolateral neocortex”, including OF, g, Aud, T, and perirhinal cortex. I’d claim that those latter regions are doing a lot of the action selection, including combining key stimuli with internal state information, and in the case of temporal cortex (T) eventually specialize in object recognition.
So in summary, the neocortex is an expanded part of a very old region that served sensorimotor control in early vertebrates, and most of it is still playing that role. When people like me record neural activity in frontoparietal cortex that’s what we find – a mixture of “sensory”, “motor”, and “cognitive” variables. But it’s not because the brain strangely combines these things. It’s because the combined thing is the thing, and has been all along.
Refs:
Kaas, J.H. (2017) “The evolution of mammalian brains from early mammals to present-day primates” (Chapter 3 of S. Watanabe et al, Evolution of the Brain, Cognition, and Emotion in Vertebrates: Springer Japan)
** I’d like to add that Mountcastle is my academic “grandfather” and one of the founders of my field, so when I say “respect”, I really mean it.
18 Likes
So wonderful to have you here!
If you want a quick introduction, I can’t suggest enough to watch this presentation by @jhawkins. It’s a great overview and the fastest and easiest way to get up to speed.
After that, the most important papers are probably these four (although there are many more excellent papers).
Why neurons have thousands or synapses - A theory of sequence memory in the neocortex
A Theory of How Columns in the Neocortex Enable Learning the Structure of the World
Locations in the Neocortex: A Theory of Sensorimotor Object Recognition Using Cortical Grid Cells
A Framework for Intelligence and Cortical Function Based on Grid Cells in the Neocortex
6 Likes
HI Paul,
Thank you for contributing to the forum. I think our views on the evolution and phylogeny of the neocortex are more similar than you think. It is clear to us that the neocortex, or if you prefer isocortex, is not a “new” organ in that is an evolved version of previous and older structures. But it isn’t exactly the same either. There are several laminar structures such as the hippocampus, entorhinal cortex, and retrosplenial cortex that have some commonality with neocortex but also vary in large and noticeable ways. What is remarkable about the neocortex, in our opinion, is how quickly it became so large. Mountcastle proposed that the neocortex got large by mostly replicating a common structure. The rapid expanse of the neocortex is strong evidence he was right about that, regardless of how the common structure evolved or how similar it is to much older structures in the brain. It is the rapid expansion of the neocortex within the past several hundred thousand years which is why we feel justified in calling it “new”.
We believe that the neocortical circuitry is a refined and repackaged version of older laminar structures. In our 2019 paper, “A Framework for Intelligence and Cortical Function Based on Grid Cells in the Neocortex”, we argue that the circuitry in a small part of the neocortex, a cortical column if you will, is a repackaging of grid cells and place cells. We argued that columns in the neocortex learn models of objects in the word using the same basic cellular mechanisms as the hippocampal complex learns models of environments. That is, each cortical column has equivalents to grid cells, place cells, and head direction cells.
Any animal that exhibits goal oriented behavior is probably using the same basic modeling circuits someplace in the brain. Under this view, learning models of the world using metric reference frames (such as grid cells) evolved a very long time ago and that something akin to grid cells exists in many brain regions in many non-mammalian species.
If you have a chance to read our 2019 Frameworks paper I would appreciate hearing your thoughts as it relates to your work.
Jeff
10 Likes
I appreciate the strict interpretation that the isocortex is the same in a lizard as in a mammal but there has to be a recognition that at some degree of development the functional arrangement of the parts has changed.
What would you chose to distinguish the relationship of the functionality between the isocortex and the rest of the brain in your basic lizard vs a human? Yes, the “other” parts have also undergone enhancement (particularly the cerebellum) but the relative mass relationship has dramatically altered.
As you move from a unicycle to a motorcycle there has to be some point where you say that the nature of the thing has changed.
5 Likes
Dear Jeff and Mark,
Thanks for your comments!
It is true that the mammalian neocortex is a big innovation, and that its rapid expansion in modern humans is important, but I don’t think the data supports the idea that it has fundamentally changed its functional organization. That primordial mammal brain I showed from Kaas maps pretty well to most mammalian species, including humans, in terms of the general topological arrangement of functions. In the highly studied rhesus monkey, there is a large expansion of prefrontal cortex at the anterior end, the posterior parietal territory between the visual and somatomotor area, as well as the posterior end (T), which expanded so much that it curved around the insula to form the familiar C-shape of primate brains with a huge temporal cortex (dragging the hippocampus and amygdala forward). Nevertheless, the regions are still there in basically the same arrangement, and those in the dorsomedial neocortex are still concerned not with knowledge about the world, but with guiding interaction with the world (For those who are interested we review the data here: [https://www.ncbi.nlm.nih.gov/pubmed/20345247]). My proposal is that most of the primate neocortex is still performing fundamentally the same thing as the ancestral pallium – it is controlling actions. What is different from our ancestors is that primates have a higher precision of actions and a much more diverse behavioral repertoire. A motorcycle is much faster than a unicycle, but it’s functional role is still to move a person from one place to another.
Now, above I’m just talking about monkeys, which is what I study in the lab. They are pretty impressive animals, but by any yardstick related to cognition humans are far superior. Watching my son learn to speak brought home just how amazing humans are. So now the question is whether our brain became fundamentally reorganized or whether it just used its size to learn progressively more sophisticated cognitive strategies. Suzana Herculano-Houzel argues that human brains are indeed large, but their organization still reflects the basic primate plan, albeit scaled up. She proposes that what gives us our cognitive advantage is simply that we have more neurons (more neurons for each type of behavior, more behaviors, and more flexibility to learn new ones). Many people have supported that view, suggesting that the neocortex expands so much because it’s a densely interconnected structure so doubling the gray matter more than doubles the white matter, and you get expansion faster (in terms of mass) than that in non-densely interconnected regions. If you compare the neuroanatomy, there are only a few striking differences between monkey and human brains (e.g. the arcuate fasciculus, linking inferotemporal and lateral prefrontal regions, is much larger in great apes than other primates). And although there are many challenges in comparing across species (especially if you can talk to one but not the other), the neurophysiological similarities between humans and other primates are overwhelming.
Given the apparent architectural similarities, some suggest that what really gave humans a boost in cognition is the ability to navigate in a complex social domain, using language, symbols, etc. I think of it as extending the domain of our control system further into the world, through other creatures. For example, a helpless baby doesn’t need to perform any complex activities to get what it needs – it simply needs to utter a sound, counting on the fact that within its niche, there are complex creatures (mom, dad) that will take care of all of the necessary complexity. They’ll get the food, call the doctor, etc. Gradually, the child can learn to fine-tune its control using different sounds (symbols) to satisfy different needs. In addition to satisfying its demands, the parents will deliberately guide the child into a rich world of human meaning (repeating their own utterances, slowing down to demonstrate stuff). With each generation that world gets richer and richer, and the brain just needs the machinery to keep up. Of course – there’s the challenge. How on earth does our cave-man brain keep up with the world into which we’re born?! I don’t know, but I do think whatever the answer is has to be compatible with the brain’s neurophysiology and its evolution.
On that note… Yes, I should read your papers to get a better idea of where we overlap in our thinking. I just downloaded the 2019 paper and will read it soon. The idea certainly sounds intriguing – there’s evidence of place cells even in the medial pallium of fish.
I wish I had fewer tasks that keep me away from the fun stuff.
9 Likes
I really appreciate you joining our conversation, Dr Cisek.
The interesting thing about the structure of cortex is that it can be used to model and simulate non-physical space. This allows agents to create new concepts by building them out of combinations of previously experienced sensations. I think the fast homogenous expansion of isocortex over the past 500K years happened because this new ability turned out to be extremely useful for survival. You seem to have a similar take on this:
Finger dexterity gave agents a need for richer representations of objects being handled, and went hand-in-hand with the expansion of isocortex. I think language had much the same hand-in-hand evolutionary dynamics with even further expansion in the non-sensory worlds of social constructions, which you alluded to previously.
6 Likes
Being exposed to the same idea but formulated differently is a good way to progressively understand the idea. Thus I was happy to find this article whose fourth part was certainly written by Paul Cisek:
Extract:
In all mammals, the neocortex consists of two sheets (Finlay & Uchiyama, 2015), a dorsomedial sector that is spatially topographic and a ventrolateral sector that is non-topographic. In primates, the former includes a medial and dorsolateral prefrontal cortex, cingulate regions, all of the premotor, motor, sensorimotor, and parietal cortex, as well as the retrosplenial cortex. The latter includes parts of the lateral prefrontal cortex, orbitofrontal cortex, and all of the limbic cortex and the temporal lobe. Most relevant to the issue of attention is the dorsomedial sector of the neocortex, which is organized into a set of fronto-parietal circuits dedicated to different classes of species-typical actions (Graziano, 2016; Kaas & Stepniewska, 2016). In early mammals (300 Mya), this system was probably quite limited and consisted simply of medial circuits concerned with locomotion and lateral circuits concerned with head and mouth movements (Kaas, 2017). Each of these circuits processed sensory information in an idiosyncratic manner specialized for its specific type of action (e.g., space near the legs for locomotion, space near the snout for ingestion) and each projected to a specific set of relevant effectors. In a sense, each circuit was an “action map” analogous to the much older tectal systems for approach and avoidance, but guiding the much wider repertoire of task-specific interactions available in the mammalian niche.
@cisekp: In your view, what would be the ventrolateral equivalent of your dorsomedial action maps? A mixture between value, salience & symbol maps associated to the current active action maps?
5 Likes
I fully agree with Dr. Paul Cisek on this point. Thank you for bringing it up, Dr. Cisek. One aspect that has not yet been included in this framework is the emergence of social interaction, also in other more primitive species, from which we evolved and of course, specially in our human episode of evolution. If we look at the timeframe slides in this paper, what is still missing in my humble opinion, is a social/anthropological dimension, not only for the past 600.000 years of human emergence, but also other earlier branches that developed social strategies like “fight or flight response” and which we inherited. This may still be a very speculative, uncharted part of our evolution, but I do not think that any of us would question its extreme relevance. Human cognition is embedded in a social process. We would not be what we are in pure, individual isolation. That is well known. In the past 600,000 years, our survival probabilities (natural selection) was largely shaped by social group fitness. This outweighed fitness of the individual. (And this probably applies also in other species and ancestors to significant extents). Our need to develop social interactions and social etiquette also formed our control of our attention, beyond the level of any other species. The development of our finger dexterity is also a product of labor-specialization, which arose with social group organization. Also the development of hemispheric specialization (right-handedness or left-handedness) is indeed believed to be a product of social evolution (primate/early human mothers, carrying child hypothesis). I think that it was this social dimension in our late evolution that lead to the explosion of our cognitive capabilities, especially after the emergence of language. Thank you very much for including this important comment, Dr. Cisek.
Joe Anthony Perez
@rhyolight
@cisekp
1 Like
I am pretty sure that some of this has been baked in from very close to the start the phylo tree.
As soon as we started to transfer genetic material from one body to another instead of broadcasting genetic material, we had to be able to recognize a mate and perform whatever goes with that act. Mating rituals, mediating mating with territorial defense, nesting, pair bonding, other related social acts.
2 Likes
Thanks for the comments, @Bitking. I fully agree with you.
The social process (or group swarming process) was probably embedded in our nature very early on. And what I find fascinating to think about is that it is most certainly an ongoing process. In the early roots and stems of the phyllo tree, the ability to survive was already highly impacted by group building and collaboration. In the primitive beginnings of life, collaboration was limited to symbiotical interactions, which never the less, established distributed information processing, across a group of individuals. Later on, some groups evolved in different directions. Some groups of single-celled organisms worked together so closely that they even became multi-celled organisms, redefining a new level of individuals composed of many specialized individual cells. Now these new multi-celled individuals also formed groups which also evolved as groups. Please note I wrote that groups evolved as such, not only individuals underwent an evolution. And these groups developed with different strategies for adaptation in different directions. Some creatures in groups evolved swarm intelligence early on. (bees and other insects and fish and birds). This probably limited the information processing ability and quantity of individuals, but allowed the group to cope much better with their specific environments. The groups abilities are always much greater than the sum of their individual abilities. Other groups developed herd behaviour. Some went down the path of optimizing their predatory skills (the fight instinct). Others optimized their herd-alertness and quick response to danger (the flight instinct). One interesting question is whether we humans inherited our social and group behaviour abilities from several of these primordial group evolutionary phases.
My guess is that group behaviour and group cognition in humans are connected to the extent, that no individual could unfold our current most complex levels of cognition in absolute isolation. Language and complex predictions in many areas would not be functional. Needless to say, social intelligence would not be refined. Our own concept of identity would not be established, except for very limited domains. We need to acknowledge that most if not all forms of life on earth, can only survive in groups, from bacteria to plants and animals. And all this life evolved as groups, not individuals for most if not all of its existence on earth. This must have left significant features in our cognitive foundations, which could only be understood if we consider them in groups, not just the physiology of individuals. In areas like ML we are already acknowledging the need for adversarial learning strategies, in just about all models. We also use GANs for output generation. These models are still far from our social-biological realities, but their authors have recognized how indispensable these group-effects are.
It should not astound us, that the process of deep grief we fall into after the loss of a very closely loved person, such as a parent or sibling, comes close to experiencing an amputation of a portion of our brain. At an abstract evolutionary level, this is practically what is happening to us. The sense of loss which we as individuals experience in such occasions, is indeed a loss of a portion of our own identity. A trusted and intertwined part of our adversarial and collateral thought process. We bounce off ideas and impressions and perceptions and body-language etc. with our trusted loved ones. Such a loss, if we consider ourselves social-beings is not just an externality. The healing process is indeed an information circuit mending process. Memories have to substitute information exchange. And this happens even at an emotional level, with extremely complex associations. For me, this experience, was a painful confirmation that we are not individual beings. We are social-beings to our very core.
1 Like