Note about the memory effects seen when you use linear mappings as associative memory.
Prior memories can hang around for a long time, including random memories programmed in by random initialization.
That’s because storing new memories only slightly damages prior memories, then it’s just a hit count of how many new memory training examples have damaged a prior memory and by how much.
https://archive.org/details/linear-associative-memory-dynamics-and-ccslm
The online linear regression people already know the math of that.
Perhaps though are less aware of the mechanistic interpretations.
One application is concatenated context selected linear mappings: https://sciencelimelight.blogspot.com/2026/09/an-introduction-to-concatenated-context.html
A person with a lot of knowledge in the linear area is Andrew Saxe. And in one paper he discovered an adjacent system to CCSLM: https://arxiv.org/pdf/2207.10430