According Russell and Norvig there are three categories of learning algorithms, distinguished by the type of feedback the algorithm receives.
Unsupervised learning algorithms receive no feedback about their performance.
Supervised learning algorithms receive the “correct” output to learn from.
Reinforcement learning algorithms receive a series of rewards and punishments in response to their performance.
Evolutionary learning algorithms don’t receive any explicit feedback, they either survive and reproduce or they don’t. I’ve included this category because it’s relevant to neuroscience.
The brain uses all four types of learning.
Evolutionary Learning
Knowledge is hardwired into the spinal cord and brain stem. This knowledge tends to be rudimentary but useful for staying alive and reproducing. Edit: I want to clarify that evolution happens via the reproductive system over the course of many generations, and that the results of evolution are manifest in the brain.
Decerebrate Cat walks and exhibits multiple gait patterns
Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis, 2011 www.pnas.org/cgi/doi/10.1073/pnas.1014269108
Review article, this is a good starting place.
I don’t know what they mean by “evolutionary learning”. If they mean changes acquired strictly by organism reproduction/survival/mutation, they do not seem to account that removing cerebrum should leave cerebellum in place.
Which has a huge role in coordinating and tuning movement I doubt it is pre-wired, it needs learning too.
Artificial Intelligence, A Modern Approach.
Editors: Stuart Russell and Peter Norvig
ISBN-13: 978-0-13-604259-4
See chapter 18: “Learning from Examples”
The cerebellum is helpful for fast and accurate movements, but is not strictly necessary for basic movement. People with missing / damaged cerebellums can still walk, although they have difficulty.
The basic sequencing of muscle movements for walking is clearly found in the spinal cord.
A simple and fun experiment that you can do at home is to turn off your cerebellum for an hour or two with alcohol. The cerebellum is affected by much lower concentrations of alcohol than most other parts of the brain so you can disable just the cerebellum with a few drinks.
That is phylogenetic learning, the parallel is Genetic Algorithms.
You might find decortication studies interesting, even with humans; i.e., the latter being people born without a cortex. These studies give a huge amount of insight into what an animal can do without one.
I know I’ve shared this before, but if you can find a hour of your time, it’s worth the watch. Essentially you give a system “genes”, which are interpreted at runtime to generate a given network architecture between inputs, middle layers, and outputs. They start out random, but given whatever selection criterion, you choose ‘survivors’ who go on to provide the foundation for the next generation, with a mix of genetic crossover and mutation.
I didn’t want to imply genetic algorithms aren’t worth pursuing in search of “intelligence”. I only doubt the decerebrated cat keeping its gait has anything to do with them or genetics in general.
The big issue with genetic algorithms is the exponential growth in search space of viable exemplars as problem complexity increases.
They have the advantage that “learning” can be trivially scaled to millions of computers but still might not be enough.
What could help would be a generic means to split arbitrary complex problems in a “chain” (or “tree”?) of several smaller problems. Each sufficiently small to be solved in reasonable time with search techniques like genetic algorithms.
Walking (on four legs, as most animals do) is orchestrated by the spinal cord and brain stem.
The neural circuits which implement walking are formed and functional before the animal is even born.
As evidence of this fact: here is another video, this time showing deer getting born and then taking their first steps within an hour of birth.
I don’t know about deer I know herbivores get up very fast. Kittens however are quite insecure, can barely rise on their legs and are far from what that zombie cat does. or “knows”.
Either way, if you argue they are born knowing how to walk then they don’t learn anything, you may call it inherited trait, not something that the newborn brain acquires through “evolutionary learning”
If there-s something they do need to learn after birth I wouldn’t count it as “evolutionary” but, as most animal learning, a form of self supervised reinforcement learning.
I’d argue that stochastic gradient descent is misleadingly as exponential; It just happens to constrain itself within the search space in a narrow, perhaps false valley. On the other hand, genetic algorithms are at least honest in what they’re doing like “Let’s make a bunch of randomized variations, take the ‘winners’, then combine and mutate them so see if it can solve a problem.”
I’m not certain of the ultimate supremacy of one over the other, except that in the case of genetic algorithms, it’s closer to how we got where we are biologically. SGD gives us the illusion of tractability, while GA makes no such claims. Of course I’ll still use both in my daily work, but at least we can be honest about what’s what.
So, intelligence is all about effectively pruning search space, the finer-grained the better. As long as pruning benefits exceed opportunity costs. Problem is, no one seems to know how to quantify such benefits at a fine grain for unsupervised learning. Just look at these meaningless negative “definitions”: “unsupervised”,
Of course they get feedback, just look at backprop in autoencoders, etc.
Performance here is always projected match, how difficult it is to understand this?
And evaluation of this performance should start from from the beginning: adjacent pixels, as in edge detection kernels, not some ridiculously distant output layer.
@MaxLee I agree there-s untapped potential in GA-s, Yet for problems past MNIST solving, all I’ve seen was just hyper parameter optimization for just another CNN.
I think you are talking about inherited value-charged patterns, AKA instincts. Yes, they are expressed in spinal cord, brainstem, hypothalamus, in lesser proportion in amygdala and basal ganglia, in still lesser proportion in limbic cortices and elsewhere.
They are “designed by evolutionary algorithm”, but don’t implement one. What they do implement is supervision, in whatever areas they are connected to. If there is an evolutionary algorithm in the brain, it’s something like “neural darwinism”, but I don’t think it actually happens. Evolution is simply way too coarse (=dumb) for the brain.
This makes total sense if you need to minimize the amount of genetic code we need to pass on to next generations. Because genetics is so hard to “teach”.
e.g. a generic useful code would be “do whatever others do”. I’ve seen it in kittens and is obvious in humans.
They might just continue their genetically-coded development after birth. I mean, they probably learn stuff too, but deer probably learn how to walk better too.
This is exactly what happens. The decorticated cat has everything but a cortex, and even then I would proffer that a small amount of it above the brainstem is left–you can only excise so much before death occurs. The point is that the complex mechanism of walking does not require a cortex, period. Now, for the cat to know which way to go, to run after a squirrel, or do other things, for that it needs a cortex.
I may have simplified this a bit, here is a better explanation and decorticate may be extreme beyond decerebrate:
Decerebration is the elimination of cerebral brain function in an animal by removing the cerebrum, cutting across the brain stem, or severing certain arteries in the brain stem.
As a result, the animal loses certain reflexes that are integrated in different parts of the brain. Furthermore, the reflexes which are functional will be hyperreactive (and therefore very accentuated) due to the removal of inhibiting higher- brain centers (e.g. the facilitatory area of the reticular formation will not receive regulating input from cerebellum, basal ganglia and the cortex).
I offer that most of the built-in behaviors are very old in an evolutionary sense and are expressed in varying degrees across the animal kingdom.
I have never seen a complete catalog of “instincts” (I have made a few half-hearted attempts to find this) but I suspect that many of these are things “we just feel like doing” to the extent that most people can’t see that this is programmed behavior.
I have observed the human nesting reflex in my wife (six times) and have discussed it with her when she was exhibiting said behavior. She was doing things that were not at all typical behavior for her. She explained that it was “what she felt like doing.” That said, she delivered within a day of exhibiting this behavior in every case.
I think that this rather large repertoire of behaviors is well preserved from very early in the evolutionary tree. Different critters express some portion of these behaviors based on internal and external cues. Minor (or even major) variations on this evolutionary learning are waiting to be tested by stressful situations - the winners pass on these behaviors by surviving in these trying situations.
This built-in programming is distributed in varying ways in different body components - evolution works like that. The other “learning types” are layered on top of or in addition to this very old evolutionary base. Like many things in the body, it is often very difficult to sort out where the dividing lines are.
