Since the perceptions itself each store information about the (likely) next transition(s), events could be encoded as the first memory only, and perhaps its context.
Thinking about the first memory of an event and about the right context would automatically trigger the right transitions which in turn would make us think about the successive events.
Therefore, successions of “scenes” could be encoded as “first scene + its context”. In other words, memories do not need to store a succession of transitions and/or as a succession of perceptions, but only the initial state. The contents of each scene (which are each encoded in its own part of the cortex) will each store the transition to the next one.
I’ve thought a bit more about this, and I think sleep may be necessary to bring intelligence back into alignment with itself.
It seems (at least biological) intelligence is necessarily distributed. Thus intelligence is also necessarily a distributed consensus phenomenon. Therefore it seems to me that as we are awake, the different aspects of our brains have a natural tendency to misalign and decay their coherence with one another. This seems like it because different areas of the brain are paying attention to different things.
It seems that sleep, where the whole brain participates in one cycle, is a way of re-aligning the frequencies and cycles of intelligence on every scale such that we don’t go insane.
Anyway, that’s not very specific or scientific, but I hope this flowery language helps someone’s intuition on this topic develop.
I don’t see how such a transfer could occur for a few reasons, primarily LTP and LTD timing would not be transferred due to the different frequency of sleep spindles. This difference in frequency would ater the synaptic positioning along the dendrite  as a preliminary issue.
Secondly, consider the SDR model whereby the patterning requires a carry over of state of prior inputs for the new learn element to be added. On a singular SDR area/column this may well work, however spindles are out of temporal sequence (in parallel, hence why we halucinate/dream) the issue then becomes a problem with distant dendritic connections which are completely out of sequence and would create incoherent synaptic connections ? (between columns / areas)
Also, when considering a delta only, I can see how LTP could be transferred easily but how would you then transfer the effect of LTD as sleep spindle timing is likely faster (and slower) than the original sensory input ? Too fast and it becomes proximal, too slow it becomes LTD.
In order to perform a transfer as such the delta’s would have to both map the dendritic connection information and relative timing ?
That said I still have no idea as to why we can’t stay awake 24x7, although I really like the idea of it’s due to coherence as that makes sense.
We may ask whether brains evolved to adapt their functioning to the general metabolic destruction/reconstruction cycle.
An hypothetical example - short vs long term memory.
Maybe short term memory works by “exhausting” inhibitory synapses with strong localized signals, which allows target neurons to become responsive to the same signal received through normal synapses.
The hypothesis being that it might be faster to “fry” a synapse then let sleep’s restorative physiology to repair it while the neurons selected for more permanent memories are “trained” to store the recent acquired patterns.
PS or even a general process by which synapses are permanently grown then destroyed either by frying via overload or by starving/forgetting
This is not a theory looking for a problem to solve, this is an attempt to explain a known process.
Note that the work in this field centers on the details at the cell level. I have been looking at this from the higher level of the nature of the engram patterns and what is special about these patterns to trigger the consolidation mechanisms described in these papers.
There are several high-level constraints that any useful theory has to explain.
I refine this with the known fact that you can recall episodes during the day before they are consolidated into the cortex. This tells me that the EC/HC index communication to the cortex is always active.
Add to this that HM and other patients can remember episodes before the loss of the EC/HC complex so episodes can be transferred to the cortex and function there.
The various engrams share the same computing fabric (A palimpsest) so the “readout” of the engrams must be ether sequential (seems unlikely) or a parallel process that transfers the “new” contents from all the engrams at the same time. How do they stay separated at they move from the EC/HC or map-2-map in the cortex? What is special about the newly trained columns that flags those bits to signal to the map on the other end of the fiber bundle?
During the training period (experience of the episode) the same bundles convey experience to the EC/HC to form the index in the first place. These same bundles are bidirectional. It is entirely possible that this is the key to pushing the “new” memories back for consolidation, aligned with where they were experienced from in the first place, with the HC time cells mediating the high-level sequence part of this playback as opposed to the strict sequential replay of the original experience at the lower hierarchical levels. Do remember that the HC has a bunch of specialist cells that do object vectors, direction, distance, orientation, and time - and possibly much more that has yet to be discovered. This all had to be developed in the hierarchy (terminating in the the temporal lobe) to be presented to the the EC/HC as an experience.
One thing I don’t get is why consolidation has to be during sleep only. Brain areas go into very similar alpha-wave micro-sleep / spacing-out state every few minutes, I would guess that’s for consolidation too. Although it may not be as effective as spindles, which seem to be unique to actual sleep. Also, HM couldn’t maintain memory for longer than a minute or so, that means HC must be consolidating it all day? Sleep maybe more specific to pruning-out memories.
I thought you agree that capture happens in the cortex too? Yes, there is that indexing thing in HC, but cortical columns probably also have reference frames. It’s hard to believe that we can’t use those fresh cortical memories until the next day, relying only on HC during the day.
This is my understanding:
1: both HC and the cortex learn during day.
2: cortical learning is STM, which must be consolidated via HC. Probably an exaggerated version of Hebbian coincidence (HC AND CC) learning, through burst mode.
3: unconsolidated cortical memory can’t last until sleep, and most likely needs to be used during day anyway.
4. thus, HC has to be consolidating CC quite frequently. probably during alpha waves.
For sleep to take so long, it must be doing some heavy lifting at the molecular/cellular level. My money is on manufacturing a resource that gets consumed during daily mental activity.
If it’s new neurons, then they also have to be wired in and the old ones disposed of. It’s easy to imagine all the other phenomena (sleep cycles, dreaming etc) being a side-effect of rewiring, not the primary purpose.
That’s my point, routine memory consolidation (STM->LTM) should not depend on sleep. Sleep involves a lot of synaptic pruning, so the purpose of sleep spindles may be to protect / rebalance important new memories during this pruning. But it’s all about connections, I don’t believe there is much neurogenenisis going on.
That paper covers a lot of hyposesses, could you be more specific?
For example, there is transfer theory of HC function: " As proposed by David Marr in his model of hippocampus-dependent memory  and supported by many experimental and clinical studies, episodic memories are transferred after acquisition from the hippocampus to the neocortex for long-term storage.".
That I think is contrary to a pointer theory: primary memories are formed and remain in CC. HC forms a more abstract copy that latter reconsolidates CC memories, this is different from forming brand new CC memories via transfer.
Really appreciate the info, lots more learning ahead…
Is it really known or is it know in a Donald Rumsfeld sense and more of a known unknown ? We know what goes on but have no idea as to how or where as per “entirely possible”.
Still to read half a dozen other papers…
I like the first paper, with the really interesting section “We also tested the possibility that memory traces are embedded to the cortical network during the state preceding sleep (e.g., wakefulness), and we asked the question how these traces would be modified by slow oscillations. We found that spike sequences of cortical neurons reflecting those embedded synaptic patterns replayed during ongoing slow oscillations and that led to further enhancement of characteristic synaptic patterns (data not shown).”
The slow wave replay I get and can agree with as to a process as this very much fits “within” my perception as to what goes on. Weather it is occuring in HC or Cortex the paper does not address at all as it theorises it could happen in either. The slow wave need would seem to be partially negated if Max’s theta cycles occur (reinforcement within the 200mS windows - which coincidentally is near the magic number 7 if looking at 25mS LTP…)
My theory on this is the parallel replay of memories creates a hallucinatory effect (what we then think of as dreaming) as the brain makes a reality from the out of temporal order memory replay. Wakeful state prevents parallel memory replay due to focus of attention limitations (winner takes nearly all). Parallel replay is then a critical requirement for sleep spindles to have an effect as it is the combination which allows for a different type of pooling to occur.
I would agree with this proposition. How can the cortex block any surprise from being learnt within a column after it has fired within the LTP window ? Sensory input has to arrive at the cortex to identify surprise.
I believe the HC learns a completely different representation to the cortex as it is a more abstract process of temporal phasing which is a function of hierarchical navigation, i.e. creating short cut paths to the right “next” palce in the cortex.
If I implied that the cortical portion formed during sleep is new memories I must have worded it poorly - that is not what I think.
My take is that the EC/HC and cortex are specialists in different kinds of learning, with the EC/HC being good at one-shot learning and the cortex more oriented at classic Hebbian learning. The cortical traces formed during the day fade at a exponential rate to some very low level latent memory. The sleep process transfers from one limited capacity learning system (EC/HC) to the higher capacity cortex by reinforcement of the weak latent engrams. I think of it like developing an image on film.
The EC/HC index part is the access to the latent cortical information that will be consolidated during the sleep cycle until the information is boosted in the cortex. Being located at the top of the hierarchy as the EC/HC is, pushing these keys back on the top of hierarchy ripples back down through the hierarchy to re-access and reinforce the engrams that make up the declarative (episodic) memories, normalizing the engram between between the two systems.
It is very likely that the combination of cycle time primes the cortex to facilitate the process over and above what happens during the initial learning process. If the “awake learning” used the same process it is likely that it would saturate as the EC/HC does if not “flushed” by the sleep process when it fills up.
It is well known that sleep deprivation causes memory errors like hallucinations. I assume that this is what happens when the EC/HC memories reach the limits of coding capacity and bleed into each other.
I should add that the cortex layer 2/3 pattern memory is different than the layer 5/6 sequence memory in this context. I see the 2/3 layers as the primary parts of the cortex that is affected by this transfer of memory contents. The map-2-map direct connections are primarily connected to layers 2/3.
In that case they won’t be accessible to the cortex during day, which is seems necessary to me. Even during sleep, signal-to-noise in such faded memory must be horrible, I don’t think the reinforcement by HC would actually work.
I see the temporal lobe as the top of the WHAT/WHERE sensory hierarchy before being presented to the EC/HC. At that general point in the hierarchy (temporal lobe) the cortical projections are sent to various sub-cortical structures.
There is a top-down motor hierarchy that originate with the sub-cortical projections to the prefrontal cortex and after several hops through frontal cortical maps, terminating in the motor cortex along the central sulcus.
The two hierarchies are broadly interconnected, with a prominent example being the two speech centers Broca’s area in the prefrontal cortex and Wernicke’s area in the posterior temporal lobe.
The motor output and sensory input areas along the central sulcus are very heavily interconnected.
So we have an ascending and descending hierarchies, connected at the top through the subcortical structures and at the bottom along the central sulcus - somewhat like the ouroboros - a circular structure.
Deciding what is the start and end presents some definition difficulties but I chose to put the boss, the sub-cortex, at the top of the sensory hierarchy. I chose to put the output driving the body as the terminus of the sense-act chain. This is, after all, why nature puts a brain in control of the body.