Grid Cell Inspired Scalar Encoder

As a side note, the reason I choose prime numbers for the basis periods was because I knew there would not be any representation collisions in the range that I gave. I’m fairly certain that the effective range (without collision) would probably be the least common multiple of the basis periods. For example:

periods: 2,4
numBits: 6
numReps: 4
sparsity: 2/6 ~ 0.33

periods: 2,3,4
numBits: 9
numReps: 12
sparsity: 3/9 ~ 0.33

It turns out that these types of numbering systems have been well studied in the context of electrical and computer engineering. They are referred to as Residue Number Systems which were formulated as an alternative to binary radix representations to address some of the issues related to the accuracy of floating point operations with finite number of bits. (See here for some reference material.) Now, in their scheme they are not using fixed sparsity representation, but rather binary encodings of the modulo digits. This allows them to use less bits for the final representation. For the purposes of HTM however, any operations on the SDR representation (whether it be mathematical or just transitional) would need to result from a learned set of weights which would bias the inputs to the grid cell representation in such a way as to cause it to shift in a specific direction and by the appropriate amount (modulo each of the basis periods).

2 Likes