Kanerva, Pentti (1988). Sparse Distributed Memory

[Bitking]

I did a search for “Kanerva” on this forum and could not confirm that.
Numenta is certainly aware of the work but I can’t see that Kanerva ever worked for Jeff.
The most relevant hits are here:

Further reading on SDRs and Temporal Memory

I came across a book from Pentti Kanerva titled Sparse Distributed Memory. It arrived today and upon looking through it I became surprised that I’d not heard of Kanerva’s work before. As the cortex is a sparse distributed memory system, and is the bases of HTM theory, Kanerva’s book goes into great detail of the subject of SDM (or ‘SDR’) so is a great read for extending the understanding and appreciation for how the cortex communicates in this simple but brilliant semantic language. Beyond tha…

Deep learning meets sparse distributed representations: The Kanerva Machine

“We present an end-to-end trained memory system that quickly adapts to new data and generates samples like them. Inspired by Kanerva’s sparse distributed memory, it has a robust distributed reading and writing mechanism. The memory is analytically tractable, which enables optimal on-line compression via a Bayesian update-rule. We formulate it as a hierarchical conditional generative model, where memory provides a rich data-dependent prior distribution. Consequently, the top-down memory and botto…

Genetically determined connections

If you look at the original STDP paper (Bi & Poo 1998) and any paper that measures postsynaptic potentials, it’s pretty clear that synapses can be and often are weighted. They also come in a huge diversity of different sizes, and this has some relationship to the evoked synaptic potential. And this is true for pyramidal neurons too (Bi & Poo was hippocampal pyramids), it’s just that we don’t know whether or to what degree the weight matters, computationally. You get a lot of power even just fro…