The nervous system has the ability to remember things in multiple time frames, from the present here-and-now through very long time frames. There seem to be different learning mechanisms for different time frames and for different structures in the brain.
There is one rule that is clearly part of the human experience but not well represented in the usual machine learning literature: Delayed reinforcement.[1]
An example that illustrates this feature is eating a dodgy meal. You eat it and everything seems all right. An hour later you get violently sick. That memory of that food is stamped somewhere in the brain and the mere sight of it triggers revulsion. Even though the conscious memory of the meal has faded there is some long-term memory of the last food eaten that receives a powerful learning negative reinforcement to stamp that food-stuff into your long-term memory. It does seem to be subcortical.
There is another similar possible mechanism that I have been meaning to research. I call it the “residue of experience” theory. Assume that each synapse goes about it’s normal response to action potentials and cooperates with other synapses on the dendrite as is normally described. Each time it fires some sort of metabolite accumulates in the vicinity of the synapse. At a later time, one or more reinforcement chemical messenger(s) could cause that metabolite to increase or decrease the strength of that synapse.