Multiplicative mutations lead to sparse systems

Evolution with multiplicative mutations leads to sparse modular systems, according to this paper:
The paper also says most biological mutations are multiplicative.


Crossover also seems to be quite similar. Multiplicative weight updates are involved according to some people.
Anyway I’m retired from the subject area so yeh…

Deep learning via multiplicative weight updates:
See the more section for the paper and code.

Blog on Multiplicative Weight Updates.

Can such ideas help with continual learning in neural networks?

Here is another interesting article on the topic, which comes to a similar conclusion: that analyzing biological networks (aka gene regulatory networks) is actually a much more tractable problem than people initially assume.

Degeneracy in the regulation of short-term plasticity
and synaptic filtering by presynaptic mechanisms
Chinmayee L. Mukunda and Rishikesh Narayanan

Key points

  • We develop a new biophysically rooted, physiologically constrained conductance-based synaptic model to mechanistically account for short-term facilitation and depression, respectively through residual calcium and transmitter depletion kinetics.
  • We address the specific question of how presynaptic components (including voltage-gated ion channels, pumps, buffers and release-handling mechanisms) and interactions among them define synaptic filtering and short-term plasticity profiles.
  • Employing global sensitivity analyses (GSAs), we show that near-identical synaptic filters and short-term plasticity profiles could emerge from disparate presynaptic parametric combinations with weak pairwise correlations.
  • Using virtual knockout models, a technique to address the question of channel-specific contributions within the GSA framework, we unveil the differential and variable impact of each ion channel on synaptic physiology.
  • Our conclusions strengthen the argument that parametric and interactional complexity in biological systems should not be viewed from the limited curse-of-dimensionality standpoint, but from the evolutionarily advantageous perspective of providing functional robustness through degeneracy.