In this research meeting, our research intern Alex Cuozzo reviews some notable papers and explains high level concepts related to learning rules in machine learning. Moving away from backpropagation with gradient descent, he talks about various attempts at biologically plausible learning regimes which avoid the weight transport problem and use only local information at the neuron level. He then moves on to discuss work which infers a learning rule from weight updates, and further work using machine learning to create novel optimizers and local learning rules.

Papers / Talks mentioned (in order of presentation):

- â€śRandom synaptic feedback weights support error backpropagation for deep learningâ€ť by Lillicrap et al.: https://www.nature.com/articles/ncomms13276
- Talk: A Theoretical Framework for Target Propagation: Talk: A Theoretical Framework for Target Propagation - YouTube
- â€śDecoupled Neural Interfaces using Synthetic Gradientsâ€ť by DeepMind: [1608.05343] Decoupled Neural Interfaces using Synthetic Gradients
- Talk: Brains@Bay Meetup (Rafal Bogacz) : https://youtu.be/oXyQU0aScq0?t=246
- â€śPredictive Coding Approximates Backprop along Arbitrary Computation Graphsâ€ť by Millidge et al: [2006.04182] Predictive Coding Approximates Backprop along Arbitrary Computation Graphs
- â€śIdentifying Learning Rules From Neural Network Observablesâ€ť by Nayebi et al: [2010.11765] Identifying Learning Rules From Neural Network Observables
- â€śLearning to learn by gradient descent by gradient descentâ€ť by Andrychowicz et al: [1606.04474] Learning to learn by gradient descent by gradient descent
- â€śOn the Search for New Learning Rules for ANNsâ€ť by Bengio et al: (PDF) On the Search for New Learning Rules for ANNs
- â€śLearning a Synaptic Learning Ruleâ€ť by Bengio et al: (PDF) Learning a Synaptic Learning Rule
- â€śEvolution and design of distributed learning rulesâ€ť by Runarsson et al: Evolution and design of distributed learning rules | IEEE Conference Publication | IEEE Xplore
- â€śThe evolution of a generalized neural learning ruleâ€ť by Orchard et al: The evolution of a generalized neural learning rule | IEEE Conference Publication | IEEE Xplore