I have a question in regards to input to the spatial pooler. As I understand it, this input comes from an encoder which holds fixed the number of on bits. While different length encodings can be used in different spatial poolers, within a spatial pooler all encodings must be the same length. Is this correct?
Ultimately, I am wondering what happens if a spatial pooler in input encodings which vary in length. In other words, instead of holding sparsity fixed so say 24 out 1024 bits in the input space are always on in each encoding, we input to the spatial pooler an encoding with a range (~20-40) bits that are on. Note, in some sense 20 bits would always serve as the base and any additional bits turned on represent additional context.
1 Like
No, spatial pooler doesn’t need its input being a “nice” SDR.
One of main SP’s purpose is to transform arbitrarily encoded data (inputs) into nicely formatted SDRs (output) with fixed size and (almost) fixed sparsity
2 Likes
One might also make the observation that a fixed sparsity level is an artificial construct meant to mimic what the brain does naturally through the intervention of cells which inhibit the activity of other cells. Thus, sparsity is a self-limiting behavior exhibited by the interplay of excitatory and inhibitory neurons in their own closed loop feedback networks.
So, to answer your original question: No, fixed sparsity is not required. However, you would be well served to implement some form of regularization or normalization in the input data so that you are not likely to get wildly varying rates of activation for inputs that fall within an expected range of values.
2 Likes