A SDR associative reverse indexer

Science Numenta Theory
1

Ok, that title sounds a bit odd.
The purpose of this experiment is to implement a “toy” SDR Associative Memory.

Traditionally associative memories are thought as content addressable databases which unlike key->value databases (e.g. hashtables or linear arrays), where an exact key match is searched against a previously stored key->value pair.

Associative Memory concept is different in that instead of exact search keys use a partially matching value to search & retrieve a set of previously stored values having a good match with the query value.

The “reverse indexer” I want to introduce here is slightly different in the sense that it retrieves fixed short keys (or uint32 IDs) from a query SDR.
Unlike “nornal” databases values are used to search for previously stored keys.
Unlike a “full associative memory” the search does not return matchin SDRs,

BUT if our indexer is matched with a “normal” key->value store it can be used as a “full” associative memory in two steps:

  1. Search value->ID indexer for a query SDR
  2. (optionally) retreive the original SDR values for the above matching IDs from a “normal” ID->value hashtable, python dict or other implementation.

Why is this “optionally”? because, in many cases, we can:

  • either have the ID itself encode semantics (as in the following mnist example usage of the indexer)
  • Use the ID as a pointer to the “original” data which was first encoded as a SDR before being stored in the associative index.

TLDR the relevant repository is the same as for the fly hash encoder introduced in a neighboring topic.

Files of interest are:

  • sdr_mem2d.py - the actual associative SDR reverse indexer. In a following message I’ll explain where “2D” comes from. The crude README on github might help partially.
  • mnist_data_loader.py, mnist_data.npz
  • fh_am_test.py - short hand for fly hash associative memory test - which does an end-to-end MNIST classification at >94% accuracy, but with much smaller encodings SDRs <2% sparse 2048 bit sdrs instead of the much larger and denser 7% 6420 bit size ones tested here
2 Likes