The power of bit pair expansion

Science Tangential Theories
4

To further expand the subject, I reached the point of two questions -

  1. why is bit pair interesting or useful, what properties makes it that way.
  2. what can it be used for.
    I shall add:
  3. what are its drawbacks and limitation.

I will begin with 2. because we already have quite a few experimental results. Other use cases might be invented.

Here is a compact enumeration of what I know of:

  • significant classification improvement when a classifier is trained on the bit pair expanded SDR instead of the original SDR (already mentioned in the first message above)

  • triadic and diadic associative memories extensively discussed in another thread
    These are SDR equivalent of either key->value or (key,key)->value repositories, having the associative property which means

    1. with imperfect/degraded/noisy versions of the keys, chances of correct value retrieval is still high
    2. even when retrieved value is different than the stored one it has a high bit overlap to the original stored value.

  • Based on the triadic memory @POv also implemented a sequential memory which predicts following SDRs in time series with promising initial results.

  • ValueMap is an associative memory that maps SDRs to a vector of scalars. So far it proved useful in two ML tasks:

    1. CartPole RL problem, where a SDR is used to encode state and values stored for different states encode a Q-value of agent’s choice of moving the cart pole left vs right.
    2. MNIST classification problem applied to the un-encoded B/W, flattened, 784 bit size MNIST image, a makeshift classifier in which a 10 value vector encode chances of a SDR being ‘0’…‘9’ digit class, plateaus at ~97.6 accuracy.
      Slightly higher results, around 97.8% are reached using a 2d or fly hash encoding

  • ID (or pointer) map. In this case the SDR is mapped to a set of potential IDs. An ID is a simple integer. So an ID map can connect SDRs to arbitrary data. This sounds a bit abstract, here are a few potential use cases:

    1. retrieving the exact most recent moments in time (time steps) when a current time SDR eventually occured
    2. a variant of near neighbor indexer, that was tested promising results in (again) MNIST digit classification.
    3. Sequential memory / next time step prediction.

Ok, I’ll pause here, I guess this is already TLDR :slight_smile: