BioHash: biologically-inspired hashing into "sparse expansive representations"

Are “Sparse expansive representations” the same as SDRs? It is interesting that the authors of this paper found a common solution, but from a different domain (fruit fly olfactory circuits).

Bio-Inspired Hashing for Unsupervised Similarity Search

BioHash is based on another paper (with Hopfield and Krotov) that introduced “FlyHash”. BioHash uses a biologically-plausible learning rule, while FlyHash does not do any learning on the dataset.

Personally, I am curious if Olfactory circuits benefitted from having a NeoCortex before Auditory or Visual sensors ever did… and if the mysterious algorithm for Visual scene understanding has a simpler implementation in the Olfactory system.

The fastest random projection/LSH you can do on a digital computer is HD where H is the Walsh Hadamard matrix (ie. use the transform) and D is a diagonal matrix with random +1,-1 entries. That is a fully connected random projection where a change in one input affects all the outputs. HDHD is better yet. Of course you can do faster not fully connected random projections.
I think you could do fully connected biological random projections very quickly using very few layers of computation because 1 neuron can connect to thousands of others. 2 layers of randomly connected neurons should be able to do a 1 million dimension random projection.

This kind of gets my ‘goat’, as to me they seem fundamentally the same thing, bar a specific constraint of being expansive, (no reason to structure an SDR thus either). So why not cite the SDR work by numenta at the very least?

Seems a bit disingenuous to me, unless i’ve read this wrong.

Looks like the BioHash paper wasn’t accepted in a conference last year, so maybe that’s karma :face_with_monocle:

1 Like