Some ISI problem?

Continuing the discussion from The evolutionary history of integrated perception, cognition, and action:

Easy, right ? ISI stands for Inter Stimulus Interval, and is sometimes accounted for in researches about pavlovian forms of learning, where often, an originally “neutral” stimulus is paired to an aversive (or to the contrary, liked) one, looking for clues that an association was learned between the two.

Most often during these experiments, the (usually) neutral stimulus above is called CS (for Conditioned Stimulus, since this stimulus is the one which, hopefully, will have formed a new association, after “conditioning”… aka training). And the innately-aversive or innately-liked one, the US (for Unconditioned Stimulus). The amount of learning will then be evaluated based on the observation (or lack thereof) of a behavioral “Response” which was known innately associated with the US, beforehand. During training, the ISI is then the amount of time between the CS and the US.

It is quite striking to me that ISI is an often overlooked problematic: for those scientific endeavors which do care about it, we may further distinguish, among these association experiments, the ones called “trace conditioning” in which there is a temporal gap between the end of the CS and the start of the US.

Depending on the species, some relevant gaps may be measured as short as 0.5 seconds, up to (I guess, but I didn’t dig too far about the longer ones since it was of less concern to me) minutes, still resulting in successful associative learning in the studied species. The original “Pavlov’s dog” realization was an example of trace conditioning indeed, and a quite long one at that.

So, what’s the fuss ?
You’ll find lots and lots of models and theories explaining how associative learning come to be on a functional level. What excites what, what inhibits what, where it happens, how to explain some second-order properties, etc. They seem to be discussed at length, and they have been for decades… While there is an interest alright in discussing those, it’s depressing that we’d still hit same papers while looking for what I’d call… “the ISI problem”.

Research about specifically trace conditioning in mammals currently hint at a fundamental role of the hippocampus (or its homologue in other vertebrates). I’d say it is not so surprising, since it is where episodic memory and measures of temporality are likely to reside, we certainly use that part for great effect there, and we’re even able to correctly predict the timing between the CS and US, provided it was consistent during training.

But mammals don’t interest me so much here.

The thing that troubles me is… associative learning can be accounted for in almost all animal phylums. Chordates alright (comprising us vertebrates), but also Nematodes (roundworms), Cnidaria (sea anemons and the like), Arthropods (insects, arachnids, myriapods, crustacean), etc.

That these organisms also exhibit learning was a recent realization for me (even if it has been known for a long time), but it shouldn’t be too surprising, considering we all have nervous systems, and our neurons are eminently plastic. What is striking however is that they also exhibit trace conditioning.

And it’s like… almost no-one cares. To the point that it is very hard for me to find conclusive online evidence for or against it (I found such conclusive evidence for insects at his point, but I’m still struggling with other lineages).

The crucial point is that, other phylums would have to achieve this without hippocampus. While it’s conceivable that another complex nav-system would be a convergent evolution for insects, and that trace conditioning is, for them also, the result of a high-level organization involving specific temporization loops, short term storages and whatnots, I’d be much more suspicious of such explanations if trace conditioning could be accounted for everywhere. I mean, sea anemons???

Cuz in this case, only explanations at a much more fundamental cell-level would make sense.
And if such widespread cell-levels mechanisms existed, how could we ever hope of building a brain replica without understanding (or for that matter, modelling or simply acknowledging) them?

Mind you… Even if evidence for such a profound issue is hard to find, I’m not guilty here of being simply one single crazy hobbyist, imagining problems where there aren’t. I’ll leave some rare scientists have the final say and finish to expose that problem more vividly than I could:

Linking behavior to underlying physiological mechanisms is one of the central goals of neuroscience, but a major impediment is the large difference between behavioral and physiological timescales. A good example of this discrepancy is provided by an experiment with Drosophila in which Tanimoto et al. (1) found that an odor becomes an aversive stimulus if it is followed by a shock during training but becomes attractive if it is presented after the shock. This finding is tantalizingly reminiscent of spike-timing-dependent plasticity (STDP; reviewed in ref. 2) in which a synapse is strengthened if presynaptic spikes are paired with subsequent postsynaptic spikes but weakened if the order is reversed. Indeed, the plot of learning index versus time interval between odor and shock in Tanimoto et al. (ref. 1; reproduced in Fig. 1B) has a shape and form similar to curves showing the amount of STDP as a function of the time interval between paired pre- and postsynaptic spikes (Fig. 1A). The problem is that the timescale is vastly different, seconds in the case of the behavioral experiment and milliseconds in the case of STDP. As in many similar cases, to account for the behavioral data on the basis of synaptic physiology, we must find a mechanism to span this large gap in timescales.
[…]
For STDP to produce net potentiation over time, pre- and postsynaptic spike sequences must be correlated in such a way that it is more likely for a pre-then-post temporal ordering to occur than a post-then-pre sequence. Such correlations require an appropriate relationship between time-dependent pre- and postsynaptic firing rates. An obvious requirement, shared by any Hebbian mechanism of plasticity, is that pre- and postsynaptic firing must overlap in time, at least to within the interval covered by the STDP window function. When two stimuli are separated by seconds, as is the case between the conditioned stimulus (CS) and the unconditioned stimulus (US) in the classical conditioning paradigm, we consider that this is not an easy requirement to satisfy.

(emphasis mine), from

2 Likes

So - even worms know cause and effect?

Perhaps it’s time to consider the neural nodes as forming stable resonant patterns, as been described in multiple places in the literature?

First hits on google search “resonant patterns olfactory bulb”:

There are numerous memory systems in humans, some preserved from earlier evolutionary steps. It is easy to be seduced by the highly regular cerebellum and hippocampus on up to the cortex/thalamus systems but the clusters of the hypothalamus point back to these older nodal cluster systems for enlightenment! These nodal systems in living critters have had more time to evolve than the cortex and are likely to have many subtle features that are unexpected in such a “simple” creature!

1 Like

I’m struggling to find conclusive evidence that they do.
Yet I was hinted that they could, and now can’t find anything to prove or disprove it. Granted, it has not been years in the searching, but first lookaround is quite saddening.

Even if it seems to be strongly fading 0.5s after offset, reading and rereading the protocols from the following paper, I’m now almost sure that some not-so-advanced individual in yet another phylum (Molluscs) is still showing a trace conditioning over timescales that our current hebbian or “STDP” models cannot account for:

I thought I had a grasp on what you consider as “resonant pattern” after reading Calvin, but I’ll now have a look at those papers. Thanks.

I’m not sure if you’re warning me about some interpretation pitfall somewhere, or to the contrary, if we generally agree on those points… but I’d rather state the following again:

I’m currently exploring the models about them, as you know.

By acknowledging, say, a performant navigation system in bees, probably evolved independently of ours, I’m well aware of the fallacy of a scala naturae, and recognize that there probably is a case for the existence of many intricate systems beyond what is found in humans.

And I don’t necessarily want to sweep away diversity, and cram all learning and modes into some single model.

What I find troubling, however, is that so many instances of that same (to me) puzzling fact : trace conditioning, is found across so many distinct lineages. Convergent evolution can happen, alright. But its probability vs the simplest explanation that the LCO had a given feature itself diminishes with each distinct branch found, sharing that same feature.

Even if that feature would appear puzzling in the context of the LCO model.

Currently the feature puzzling me is the retainment of sensory information long enough for anything like trace conditioning to happen. As some (unfortunately mammal-study) paper would put it: Where is the Trace in trace conditioning?
And the LCO model (basal “ParaHoxozoa” ?), although very fuzzy, is most certainly very low in its nerve-cells count, scala naturae or not. Wikipedia would have that being around 680mya… That’s almost 150 million years prior to the start of the cambrian explosion.

At this point today, were we to find a single paper about unmistakable trace conditioning in sea anemone, that would seal the debate for me: We’d need to find a fundamental learning mechanism of all nerve cells (or almost-any-simple nets of them) beyond STDP.

Not so much a warning as expressing caution in discovering that the animal model does not bring as much enlightenment as you may have hoped as it is not a perfect single thing.

In your list of critters you mentioned both nematodes and insects. The same way that eyes have evolved in many different lines and with a diversity of implementation details, I expect that the connection to basic chemical signalling to do spike timing may be similarly varied.

The overall scheme of clusters of neurons forming resonant patterns that engage the lower-level spike timing to be a constant that has several exemplars. In my reading I tried to jump on the first thing I saw and apply it to the next case I read to make it easier to understand. After some confusion I found that there were some minor structural differences that had be be considered.

The current mamillary olfactory bulb is not the “standard” model of resonant clusters, but more of a sign post to a general concept that seems to be found everywhere.

Plainly, a memory of a state, followed by a good or bad stimulus, forming an association. The state memory is a “short term” resonant pattern that reliably stands for the state. The evolutionary innovation is the extension of spike timing past the very short window of normal spike timing by holding a prior state until it is needed.

I should add that this remembered judgement is then used in goal seeking - often navigation.

1 Like

That’s close to the heart of my internal struggle, yup.

Some complex and intricate, carefully wired “memory system” as a specific organization of some cells, is hard to reconcile with the idea of a mother being living 680mya, especially considering that its extant offspring don’t seem to have very common such specific organizations across lineages.

So, if we take the view that such a 680my-old being already supported sensory “retainment” without exquisite mem systems (1), we need to find an origin for it within single cells, or synapses, or within their immediate ionic environment, or from very very easy-to-reproduce properties of any bunch of them “resonating” as you seem to propose.

Any way you look at it, if you agree with hypothesis flaggued as (1), looks like any NN model, even based on current animals or humans, would have to take that “STM-forming” property into account.
And we (HTM and most others) currently don’t.
Simply is my point, really.

(And I know about the multiple evos of the visual systems and it doesn’t ring same “wierdness” bell, currently, in my view. There’s something there, with that learning issue).

(Ah, and happy forum-birthday :partying_face:)

If you look at very early chord development there is a turn-around from sensory to motor directions.
Once there was tinkering to make this connection one of the early evolutionary experiments is the enlargement of that area - a cluster of neurons (ganglia) that could raise some early form of memory.

Easy example:

I am sure that there were earlier, less capable, versions.

This is one of the early papers that got me to thinking about what is going on in ganglia:
Modeling the olfactory bulb - coupled nonlinear oscillators - LI, Hopfield - 1988
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.898.2047&rep=rep1&type=pdf

1 Like

I may be oversimplifying the problem (I haven’t dug into the papers yet since I am currently fixated on a different topic), but it seems to me that reciprocal activation between populations of neurons in some sort of loop (or “daisy chaining” as someone on the forum used to talk about) for holding onto representations for extended periods of time, can be a basic enough arrangement that evolution could have converged on it many many times. Not ruling out that there is some common ancient system that has survived and been reused throughout many phylum, but in my mind I do not think it is necessary.

1 Like

Maybe, yes.
Okay. You’ve just given me an idea on how to rephrase the problem - or reorganize our thoughts.
That “basic enough” part is precisely what I find itching. Now, two possibilities, if we take the view that it is indeed basic.

  • Either it is somewhat basic, but evo has to take some slightly further step to make it happen.
  • Or it is so basic that it is actually a core building block of neural systems and you’d find them emergent at every corner of any (biological) NN.

In the first bullet, it would support the case for convergent evolution. I don’t know at this point what to make of it. It really seems (more so than, say, eyes) widespread across lineages not having a similar neural architecture. And contrary to eyes which can be wired at a single place, you’d have to decide (evo-select) to add that “basic-but-still” loop to each and every sensor, just in case.

In the second bullet, you’re facing same dreadful ISI problem as I am, to the letter.

2 Likes

Perhaps the diverse zoo of inhibitory neurons is indication that this is the case (i.e. excitatory neurons left alone will form a continuous feedback loop without “traffic cops” to keep them in check)

2 Likes

I have read (but don’t have a ready reference) that neural tissue in culture spontaneously starts to oscillate. I am going with the second choice - it is basic that if you have some number of cells they will try to oscillate all on their own, and as Paul offers, the inhibitory interneurons are the evolutionary provided traffic cop to regulate this behavior into something regulated and useful.

As far as your timing issue - if this oscillation is coupled with some short and long term memory mechanisms and some related learning paradigms then you can get the “this and then that” coupling with an extended time between the “this and then that” presentation.

1 Like

I don’t want to make a habit of cutting your quotes, Mark, but I apologize that this part is resonating with my current ideas.

Nobody really has a decisive clue, for human very-short & working mem, right ?
so, err…
“regulate this behavior into something useful” yeah maybe.
“oscillation is coupled with some short […] term memory” yeah ?

maybe? :slight_smile:
maybe there’s really something to ponder about there.

dunno, just brainstorming at this point. Sorry if I’m advancing anything silly.

Un, short term as in the actually resonant pattern and possible metaboic residue of this action, and long term as in the change in connection strength as the pattern is hit with the second positive or negative reinforcement.

A far as “Nobody really has a decisive clue, for human very-short & working mem, right?”
I think that there are many memory systems at play in the brain. For these older parts of the brain I will offer the mechanism that is most appropriate to this level of wiring:

The resonant pattern is the combination of presented inputs and internal connectivity. If you look at the Hopfield networks and Boltzmann machines you will find that they are inspired by this configuration in biology. This is not the model in Cortex so it does not get as much love but as I have said in other posts - when the focus turns to subcortical structures these types of configurations should get a lot more attention.

A video that originally resonated with me on the idea of a basic unit of working memory was Stanley Heinze’s theory about the insect central complex (at the time I was exploring path integration).

That particular video doesn’t go into the specific micro-circuitry of the working memory itself (though he does mention reverberating activity between 15 or so cells IIRC). But it does demonstrate how inhibition (from head direction cells in this case) can be used to control the activity of the working memory units in order to apply it to a useful function. In this case, the head direction cells are essentially playing the role of controlling credit assignment.

2 Likes

I had begun to watch it not so long ago but have been interrupted. That vid is quite something indeed.

Mark, I don’t feel I have the prerequisites to follow you with that last post. Nevermind. I needed to think about all this some more before having something intelligible to say, anyway.

Cheers, mates.

1 Like

It is quite humbling to realize what can be achieved by nature with just a handful of neurons. It does make me wonder if we can ever hope to understand what 100 billion of them are really doing.

1 Like

Wow! Thanks! I didn’t know much about insect brains.

Their 3000-neuron “central complex” is very conserved between different insect species (same number of neurons, same connectivity). Interestingly, the lecturer in the video estimates their common ancestor having this exact same “central complex” at at least 240 millions years ago!

Those insects can do impressive goal-directed navigation when they return back to their nest in a strait line. But that require a “working memory” of path integration. As Paul mentioned, Dr Heinze speculates that this is done through reverberating circuits to keep up activity, but the mechanism is still unknown today.

In the video and the associated paper*, it is inferred that a dozen memory units each encode the accumulation of distance in a given direction via synaptic plasticity in a 18-neurons recurrent network.

But the sustained firing activity necessary to keep the memory during hours seems to me like an overkill and energy-intensive solution. Couldn’t it be simpler via non-synaptic plasticity? At each step in a given direction, the presynaptic firing could cause the cells to accumulate some electrolytes, facilitating or diminishing their exitability. However, I don’t know if the accumulation of electrolytes can mirror reliably the accumulation of distance during hours, and if it is possible to reset easily the stock of electrolytes when the insect comes back to its nest… Possible? I need to have a look in the book “The Evolution of Memory Systems” mentioned by Guillaume!

*The insect central complex and the neural basis of navigational strategies, Anna Honkanen, Andrea Adden, Josiane da Silva Freitas, Stanley Heinze, 2019: https://jeb.biologists.org/content/222/Suppl_1/jeb188854

3 Likes

I wanted to dig further the non-synaptic plasticity mechanisms and found this very informative and provocative paper:

Is plasticity of synapses the mechanism of long-term memory storage? Wickliffe C. Abraham, Owen D. Jones & David L. Glanzman, 2019:
https://www.nature.com/articles/s41539-019-0048-y

Key points:

  • Whether or not memory is necessarily stored at synapses is still unclear
  • Cell-intrinsic engrams could also be encoded via epigenetic storage of information (DNA methylation is a good candidate for this mechanism)
  • There are good chances that both synaptic and epigenetic mechanisms are at play for our memories

Extract from the article:

Holliday’s hypothesis that DNA methylation might subserve memory has received striking confirmation during the past decade. Studies in mammals and invertebrates have documented roles for DNA methylation in the formation of a variety of forms of learning and memory.94,95,96 These studies have shown that inhibitors of DNA methyltransferase (DNMT) block the formation and/or consolidation of memory. In addition, extensive DNA methylation changes have been documented for hippocampal-dependent fear conditioning in the brains of mice and rats; these involve both hypermethylation and hypomethylation (see below) of genes.97,98 Moreover, the pattern of DNA methylation changes alters over time, with some patterns apparently associated with long-term maintenance of memory, because they occur weeks after training. Note that because DNA methylation is commonly associated with gene silencing,99 the late DNA hypermethylation observed in these studies suggests the persistence of some forms of memory requires the ongoing repression of one or more genes. Further support for this intriguing idea has come from two studies, one in rats100 and the other in Aplysia,96 which found that DNMT inhibitors disrupt well-consolidated memory

@gmirey: This form of memory is probably very ancient and could be an answer to your ISI question.

Here is another extract related to your question:

Additional evidence in favour of the cell-intrinsic hypothesis of memory storage comes from a study by Johansson et al.,87 who investigated how cerebellar Purkinje cells acquire information about the temporal interval between two conditioning-related stimuli. Their training protocol consisted of paired stimulation of the parallel fibers and the climbing fibers, the conditioned stimulus (CS) and unconditioned stimulus (US), respectively. Paired CS-US stimulation produced a reduction in the number of spikes evoked in the Purkinje cell by the parallel fibers; this reduction is believed to be due, at least in part, to associative LTD of the parallel fiber-to-Purkinje-cell synaptic connections.88 In their study, Johansson et al. carried out training using different intervals between the CS and US. Their data indicated that the memory of the trained CS-US interval could not reside in either excitatory or inhibitory networks within the cerebellum, but, rather, must result from mechanisms intrinsic to the Purkinje cells

And a last one with snails to show that it doesn’t only concern animals with a cerebellum:

Memory transfer via RNA
The strongest challenge to date to the synaptic plasticity hypothesis of memory storage comes from a recent study by Bédécarrats et al.,114 who reported successful transfer of memory from a trained to an untrained animal via RNA injection. In their experiments, which were performed on Aplysia, Bédécarrats and colleagues gave one group of animals training on two consecutive days with a series of electrical shocks delivered to their tails; the training induced long-term sensitization, a non-associative form of learning, in the animals. The sensitization was expressed as an enhancement of the siphon-withdrawal reflex (SWR) to a weak touch delivered 24h after the last episode of tail shocks. A control group of untrained snails exhibited no reflex enhancement when tested at the same time. Following the behavioral tests, central ganglia were dissected out of the trained and untrained animals, and the total RNA was extracted from the ganglia. The RNA was then purified, after which the RNA from the trained animals (hereafter trained donor RNA) was injected intrahemocoelly into a group of untrained recipient snails, and the RNA from the untrained animals (hereafter untrained donor RNA) was injected into a second group of untrained recipients. When tested 24 h post-injection, the animals that received the trained donor RNA exhibited significant enhancement of the SWR, whereas the recipients of the untrained donor RNA did not

2 Likes

Digging towards the smallest scales… like ions, proteins and, in that case, DNA… is like entering very very shady areas for my current knowledge.

At first glance, about first quote, I’d say that preventing DNA to work and claiming that it is thus “DNA, not synapses” subtending anything sounds kinda silly… but given my disclaimer above, I just can’t say “silly”, right ?

Second quote : As a matter of fact, the Purkinje cells are also outside my expertise, but contrary to the above, I’m planning to study that in the near future. Thanks for those refs.

Your last quote seems to indicate that I may have misinterpreted the first. Wait… that transfer thing is really surprising. Hey finally those pulp SF stories where some guy is cloned with retained memories could be a thing ? :wink: