I remember reading from an article or watching a video about the brief coverage of the neuroscience behind permanence of synapses. It was a great insight to learn that synaptic strength was not a representation of weight, but rather a representation of permanence. It seems this is a new paradigm as LTP still suggests that synaptic strength acts as synaptic weight.
Are there any papers that cover this topic? I can’t seem to find anything online so far.
I think if you key your search on synaptogenesis you’ll find discussions of that. The concepts of synaptogenesis and permanence are intertwined I believe?
I’m not sure if they are. When studying synaptic plasticity the main focus is on the addition/removal of post-synaptic receptors - in which results in greater/weaker post-synaptic depolarization. However this doesn’t seem to make complete sense knowing that its not individual synapses that contribute to depolarization but mainly synaptic integration zones (segments). However it makes perfect sense if post-synaptic receptors act as the ‘binding glue’ resulting in how permanent the synaptic connection is.
I assume that synaptic permanence is a modern paradigm, kind of like the modern-ish studies of synaptic integration zones. So for that reason I guess that’s why I’m having a hard time trying to find neuroscience material on it.
In Jeff’s talks, he often makes specific mention of synapses grouped together as being “coincidence detectors”, and he’s said that it only takes between 8-20 of them, spaced 20 microns (<-- That is from memory so may be inaccurate. “Close proximity” is what we’re concerned with though). If you’re looking for the direct causal reason why synapses get removed, I have never seen among the papers Numenta has released, any discussion of the chemical; behavioral; or biological causes of a synapse being removed…, so I don’t know.
Anyway, you may find these references useful:
As excerpted from the (Hawkins, Ahmad) paper: “Why Neurons Have Thousands of Synapses,…”:
Active dendrites suggest a different view of the neuron, where neurons recognize many independent unique patterns (Poirazi et al., 2003; Polsky et al., 2004; Larkum and Nevian, 2008). Experimental results show that the coincident activation of 8–20 synapses in close spatial proximity on a dendrite will combine in a non-linear fashion and cause an NMDA dendritic spike (Larkum et al., 1999; Schiller et al., 2000; Schiller and Schiller, 2001; Major et al., 2013). Thus, a small set of neighboring synapses acts as a pattern detector. It follows that the thousands of synapses on a cell’s dendrites act as a set of independent pattern detectors. The detection of any of these patterns causes an NMDA spike and subsequent depolarization at the soma.
You could try some of the references in this paper to see if there’s any mention of the measurement of “longevity” of particular synapses, which is what I’m guessing you’re interested in?
I’ve always considered the term “Permanence” to be a Numenta-born term which relates the concept of synaptic adhesion strength to a parameter which models that adhesion in code modeling. In which case, (I believe) the parameter is 2-fold: 1.) Expressing both the “strength of adhesion/degree of ‘coupled-ness’” together with, 2.) the threshold beyond which a series of NMDA-Spikes are generated which depolarizes the post-synaptic cell.
Yeah I think “longevity” is a good word. As it happens I’ve been looking through the references from “Why Neurons Have Thousands of Synapses” but have not found a direct reference for synaptic permanence. But I’ve been looking through some papers regarding synaptic integration zones (which “Why Neurons Have…” has lots of references for). Although not one of those references, this paper seems good so far. I’m trying to find reference to something similar to synaptic ‘permanence’/‘longevity’ in there.
Nice! This looks interesting so far…
May I ask, why you’re interested in the exact mechanisms behind the concept of synaptic “Permanence” / “Longevity”? Are you thinking there may be a subtlety of that distinction which could yield some additional efficiency when modeled in code? I’m not sure why this has captured your interest so? (I’m being nosy…) 
When I generally read about plasticity (long-term potentiation in particular) it seems to suggest the same model as ANN point neurons - that the soma performs linear summation of all active synaptic potentials (weights == quantity of receptors). This might be true for non-cortical neurons, but as we know - pyramidal and stellate neurons perform non-linear summations on their dendrites. So the non-linear NMDA spike would feedforward the same potential regardless of the number of receptors on each of the active synapse. This is why it makes much more sense that receptors are like ‘glue’ - the more glue the longer and stronger the connection. (apologies for the repetition, I’m still trying to clarify it fully)
So I just want to find some references so I can clear this confusion. Is it that case that the classical and the modern paradigms conflict? Or do they support each other? I suppose its just going to bug me until I clarify the details.
… So far I’ve identified 3 possible causes of “longevity” - at least as it is identified from that paper…
Synaptic plasticity involves numerous kinases, phosphatases, aswell as various molecular signaling pathways (Citri and Malenka,2008), the activation of which may be spatially constrained. Thissuggests that molecular signaling cascades may underlie thecooperativity effects observed in plasticity induction withinnearby sites in dendrites (Bhalla, 2011; Govindarajan et al.,2011; Harvey et al., 2008; Murakoshi et al., 2011). In addition, thePRPs required for plasticity can be synthesized by the proteinsynthesis machinery existing in the cell soma, or they may betranslated locally by ribosomes which exist in dendritic arbors.
====
This evidence suggeststhat dendrites may support local forms of plasticity that do notdepend on transcription or somatic protein synthesis by sustainingtheir own protein synthesis which is triggered by local signalingpathways. Dendritic protein synthesis was first identified to be arequirement for rapid synaptic potentiation under exposure toBDNF (Kang and Schuman, 1996) and has since been found to berequired for many forms of synaptic plasticity (Sutton et al., 2006)
'2. The role of inhibitory neurons in generating plasticity in excitatory neurons…
Inhibition plays a major role in shaping neuronal outputthroughout the brain, and displays significant variability in itsmagnitude and targeting (Klausberger and Somogyi, 2008). Forexample, dendritically-targeted inhibition regulates the input–output-transformations in CA1 pyramidal cells and increases thethreshold for dendritic spiking
=====
The coordinated plasticity of excitatory and inhibitory connec-tions has been suggested to play a major role in the stability ofsimulated cortical networks, where a ‘‘detailed balance’’ ofexcitation/inhibition is required (Vogels and Abbott, 2009)
'3. Regulation of synaptic plasticity by local homeostasis
“5.” Regulation of synaptic plasticity by local homeostasis
Apart from inhibition, homeostatic plasticity is another majorbalancing mechanism which acts continuously to regulate synap-tic plasticity in the long term. The effect of homeostatic regulationon synaptic clustering and dendritic excitability is thus critical in amodel of memory formation where dendritic branches play a keyrole.
Nice!
It looks as if the classical & modern paradigms might support each other if I’m understanding this correctly. I’m still digging through.
NOW I see what you’re aiming at.
What is meant by, and what is the “thing” that is non-linear?
A. Is it that the proportional number of synapses is not directly related to strength of depolarization?
–or–
B. Does it express the disproportionality between the electro-chemical charge of an NMDA-Spike and the resultant strength of depolarization?
–or–
C. Does it express something else my unfamiliarity with the topic hasn’t touched upon? 
In either case, I think that the co-occurrence of multiple causes for synaptic plasticity pretty much provides anecdotal evidence at least, for the lack of alignment between ANN-point neuron architecture and modern complex neuron architecture?
I think A. So of course a synapse in a point neuron may have a weight of 0.5, so therefore contribute to the soma depolarization of 0.5 (linear). But a cortical synapse may have a ‘weight’/‘permanence’ of 0.5 but the depolarized contribution to the soma is disproportional due to the all-or-nothing (non-linear) NMDA spike.
Well thank you so much for introducing me to that paper and that site, for that matter. I think this is an extremely interesting topic, and I’ll be watching for contributions to this discussion by more learned individuals than I.
Also, I’m going to make another attempt to get some sleep tonight / day? 
One more thing. I don’t think I payed enough attention to your theory here. But I believe that the author of the paper you linked, supports your theory (even though they discuss it more in terms of “clustering” and its affects). So yeah - I would be surprised if the longevity is not, to some degree at least, influenced by the idea of “receptor attraction” or “glue” - you are probably on to something!
It might be the case that both models are in alignment if the point neuron was augmented to include dendrite spikes. In that case the ‘weight’ of the synapse (the propagated signal strength) really doesn’t matter because the NMDA spike is a all-or-nothing activation - essentially normalizing the signal strength to the soma to provide a normalized depolarization. This may then mean that the number of receptors on the post-synaptic synapse just functions as the ‘binding glue’ (that then represents the permanence/longevity of the connection). The less number of receptors, the more fragile the connection. The greater number of receptors, the more robust the connection.
Principles of dendritic integration in CA1 pyramidal neurons
I didn’t follow all the questions but I can answer a few things related to terminology and how we view some of these issues at Numenta.
First a few “facts”
Our use of the word “permanence” might be different than other people’s use of the word. We use it to describe the growth of a synapses from “0 = not existent but possible” to “0.3 = a connected synapse with a thin spine, not very permanent” to “1.0 = a synapse with a thick spine and more permanent”.
My understanding is the there are several mechanisms that affect the growth and permanence of synapses, it is pretty complicated. Our model is simple, but it is also one of the only ones that incorporate active dendrites and synptogenesis.
Let me know you have a specific question.
Jeff
Thanks Jeff, that was exactly the kind of answer I was hoping for!
I think this description has pulled it together for me.
This may be slightly off-topic, but I think related nonetheless. What is the chemical/biological mechanism for increasing the rate at which synapse becomes more (or less) permanent? In particular, I’m thinking of the occurance of extreme stimuli or traumatic events to ‘etch’ specific patterns into more permanent memories. I always presumed that it had something to do with the presence of adrenaline or other hormones/neurotransmitters, or perhaps strong signaling from the emotion centers in sub-cortical regions.
Neurobiology is more of a lifelong curiosity than a vocation for me, so I admit to having a broad but somewhat shallow understanding of the lower level details. Still, I would be very interested to know about the underlying factors determining the rate of increase or decrease in synaptic permanence.
On a related note (which I will take to another thread if necessary), does anyone have any thoughts on how to implement this as part of the CLA? One would presume that this behavior depends on either some external physiological mechanism not currently represented in the CLA/HTM framework, or would wholly depend upon some level of pattern recognition (of danger, or perhaps novelty) from the network itself to signal the need for synapses to be adjusted more strongly.
There are numerous mechanisms that can invoke synaptic growth and change the permanence of synapses. As you point out, traumatic and other emotional events can cause sub-cortical regions to release of “neuromodulators”. The neuromodulators are usually released broadly in the neocortex and can cause permanent imprinting of important events. There is a large body of literature that describes these mechanisms.
We don’t model these kind of neuromodulator mechanisms today in HTM theory. If we wanted to, we would probably have to create a more sophisticated state for synapses and/or more sophisticated learning rules. In the kind of machine learning applications we are most interested in today we haven’t found a need for these mechanisms. It is easy to imagine adding them. For example, if we wanted to make sure that some event was never forgotten we could add a “lock” bit to each synapse. A locked synapse would not be subject to the normal Hebbian learning and forgetting mechanisms. Locked synapses would become lifelong memories…
Thanks for the explanation.
Sorry for “hijacking” the thread but I have a related “conceptual” problem: It is biologically plausible the current forgetting model in CLA?
The problem I have, is how punishment in the current CLA model relates the biology. According this code a predicted but not active cell will imply a punishment to the “optimistic” active synapses.
I couldn’t find any bio reference for that. My intuition is that biological systems forgets by repurposing synapses (i.e. use old unused synapses by a different more recent sequence). Instead of punish it looks like use limited segments and evict the LRU synapse (when we run out of them in the segment). In synaptogesis code this also seems to be not considered.
