Are “Sparse expansive representations” the same as SDRs? It is interesting that the authors of this paper found a common solution, but from a different domain (fruit fly olfactory circuits).
BioHash is based on another paper (with Hopfield and Krotov) that introduced “FlyHash”. BioHash uses a biologically-plausible learning rule, while FlyHash does not do any learning on the dataset.
Personally, I am curious if Olfactory circuits benefitted from having a NeoCortex before Auditory or Visual sensors ever did… and if the mysterious algorithm for Visual scene understanding has a simpler implementation in the Olfactory system.
The fastest random projection/LSH you can do on a digital computer is HD where H is the Walsh Hadamard matrix (ie. use the transform) and D is a diagonal matrix with random +1,-1 entries. That is a fully connected random projection where a change in one input affects all the outputs. HDHD is better yet. Of course you can do faster not fully connected random projections.
I think you could do fully connected biological random projections very quickly using very few layers of computation because 1 neuron can connect to thousands of others. 2 layers of randomly connected neurons should be able to do a 1 million dimension random projection.
This kind of gets my ‘goat’, as to me they seem fundamentally the same thing, bar a specific constraint of being expansive, (no reason to structure an SDR thus either). So why not cite the SDR work by numenta at the very least?
Seems a bit disingenuous to me, unless i’ve read this wrong.
