What are the known learning rules?

Community Lounge
1

What are the known learning rules? Is there a review, paper, document? Thanks.

1 Like
2

This is basic neuroscience and there is not much point in me rehashing some already excellent explanations. Heres is some light reading for you:

http://uhaweb.hartford.edu/compsci/neural-networks-definition.html

https://www.tutorialspoint.com/artificial_neural_network/artificial_neural_network_learning_adaptation.htm

Your self-guided tour:

Happy reading and good luck with your studies.

2 Likes
3

The nervous system has the ability to remember things in multiple time frames, from the present here-and-now through very long time frames. There seem to be different learning mechanisms for different time frames and for different structures in the brain.

There is one rule that is clearly part of the human experience but not well represented in the usual machine learning literature: Delayed reinforcement.[1]

An example that illustrates this feature is eating a dodgy meal. You eat it and everything seems all right. An hour later you get violently sick. That memory of that food is stamped somewhere in the brain and the mere sight of it triggers revulsion. Even though the conscious memory of the meal has faded there is some long-term memory of the last food eaten that receives a powerful learning negative reinforcement to stamp that food-stuff into your long-term memory. It does seem to be subcortical.

There is another similar possible mechanism that I have been meaning to research. I call it the “residue of experience” theory. Assume that each synapse goes about it’s normal response to action potentials and cooperates with other synapses on the dendrite as is normally described. Each time it fires some sort of metabolite accumulates in the vicinity of the synapse. At a later time, one or more reinforcement chemical messenger(s) could cause that metabolite to increase or decrease the strength of that synapse.

[1]https://www.princeton.edu/~yael/Publications/Niv2009.pdf

4

I like to think in term of evolution for the sequence of abilities gained. One of the first must be our position in the cycle of the day. Just as the grid cell system orients us to place there must be a circadian system that orients events to their place in the day.

Remembering poisonous food may even per-date a time system and must be highly conserved.

1 Like
5

Makes sense, but there has to be a difference between the internal clock and the external one, if there is one. And currently we encode time data into the distributed input representations separately. And then relative time can be deduced from when an event happened with respect to other events and basic logic.

Time perception is handled by a highly distributed system involving the cerebral cortex, cerebellum and basal ganglia. One particular component, the suprachiasmatic nucleus, is responsible for the circadian (or daily) rhythm, while other cell clusters appear to be capable of shorter-range (ultradian) timekeeping. There is some evidence that very short (millisecond) durations are processed by dedicated neurons in early sensory parts of the brain.

Regarding learning rules, there are good papers referenced in the HTM bibliography on Hebbian learning and other details.

6

On the shorter end (shortest?) of the timescale:

7

This paper seems interesting:
https://arxiv.org/abs/1802.02678

8

Some parts of the brain have ramping pre-movement activity which is sort of a time signal. This might be the same phenomenon as remapping, which is involved in coordinate transformations but possibly only egocentric ones. Here are some sources, some only indirectly relevant.

Neural antecedents of self-initiated actions in secondary motor cortex (Murakami et al., 2014)
Predictive Activity in Macaque Frontal Eye Field Neurons During Natural Scene Searching (Phillips and Sagraves, 2010)
Composition and Topographic Organization of Signals Sent From the Frontal Eye Field to the Superior Colliculus (Sommer and Wurtz, 2000)
The relationship of monkey frontal eye field activity to saccade dynamics (Segraves and Park, 1993)
Competing Neural Ensembles in Motor Cortex Gate Goal-Directed Motor Output (Zagha et al., 2015)
Potassium Channels Control the Interaction between Active Dendritic Integration Compartments in Layer 5 Cortical Pyramidal Neurons (Mark T. Harnett, Ning-Long Xu, Jeffrey C. Magee, and Stephen R.Williams, 2013)
The time course of perisaccadic receptive field shifts in the lateral intraparietal area of the monkey (Kusunoki and Goldberg, 2002)
Robust neuronal dynamics in premotor cortex during motor planning (Nuo Li, Kayvon Daie, Karel Svoboda, and Shaul Druckmann, 2016)
A disinhibitory circuit mediates motor integration in the somatosensory cortex (Lee et al., 2013)
Division of labor in frontal eye field neurons during presaccadic remapping of visual receptive fields (Shin and Sommer, 2012)
Parietal Area 5 and the Initiation of Self-Timed Movements versus Simple Reactions (Maimon and Assad, 2006)
Putaminal Activity for Simple Reactions or Self-Timed Movements (Lee and Assad, 2003)
Parallel processing of serial movements in prefrontal cortex (Averbeck et al., 2002)
A Map of Anticipatory Activity in Mouse Motor Cortex (Chen et al., 2017)
Maintenance of persistent activity in a frontal thalamocortical loop (Guo et al., 2017)
A motor cortex circuit for motor planning and movement (Li et al., 2015)

I’ll clean this up and describe the sources when I have time.