New scalar encoder implementation: minimalism vs ambiguity



Right now I’m trying to finish a scalar encoder. For some reason, I wanted to minimize bits needed for representation while keeping sparsity.

Anyway, here it is:


It starts with zero off bits mixed in with the on bits and slides those mixed bits back and forth over the total bits. Once it hits a side, either the permutation of the on/off bits is increased by one, or, if the mix has reached the max permutation, the number of off bits in the mix is increased by one.

I still need to debug it a lot and make it nupic compatible, as it should be capable of larger numbers when using three or four on bits and still keeping sparsity, but there’s some math error somewhere right now that’s rounding up instead of rounding down. There’s another problem though: since the on bits move back and forth, completely different numbers turn on the same columns. One way I was thinking of improving this was adding on number encoders to the main number encoder that would represent the number of off bits mixed into the permutation, as well as the permutation number. That way, those neurons or nearby neurons would stay active for larger groups of numbers. I could repeat that cycle until some neurons are active for nearly half the numbers.

So, would that ensure the scalar encoder I’m making had good properties? Am I missing anything here? This is my first time writing an encoder, so I welcome any help.


I’m a little concerned by this statement, because we don’t start representing columns until Spatial Pooling occurs. The encoders run to provide input to the spatial pooler. An encoder only represents an N-dimensional space (usually 1, unless there is topology). Your visualization above also looks like you are representing columns as output of the encoder.


Oh yeah, that was the code I was using to test temporal memory. To be honest, I did forget about the spatial pooler for a moment… :neutral_face:

Well, it’s still useful for testing. Also, I wonder how it will perform compared to the spatial pooler run on default encoders. It does maintain sparsity now.