From the BaMI SP pseudocode:
From what I’ve seen in every NuPIC example so far is that no stimulusThreshold is specified to the spatial pooler, so the default value of 0 is used, negating any of its effects:
https://github.com/numenta/nupic/blob/master/src/nupic/research/spatial_pooler.py#L120
Unless manually manipulated, the swarming process never permutes stimulusThreshold, and therefore never returns any value for it. So all models created using params returned by swarming will use the default value of 0.
I understand that this parameter is supposed to help reduce noise, but the problem seems to be coming up with a good value for it. An HTM user won’t necessarily know what values will be valid for stimulusThreshold until they have inspected a running spatial pooler and analyzed the overlap scores being generated for columns over a particular input space.
Is there any value to exposing this parameter? Can anyone explain when and why it would be used?
It seems to me that the noise that this ‘stimulusThreshold’ would most
likely be reduced reduced by the SP inhibition step anyway. With no
stimulus threshold, there could be a lot of SP columns that get non-0 but
still relatively low overlap scores which would not be high enough to make
them into winning SP columns, so these columns would just be inhibited
regardless.
For example, let’s say the ‘stimulusThreshold’ 6, but the number of active
synapses on a segment required to make the top 2% and become a winning
column is 8. In this case many columns might have overlaps of 1 to 5 which
would be turned to 0 from being under the threshold, those these columns
would be inhibited anyway since they were less than eight, so the threshold
isn’t ultimately making a difference in what columns are chosen as winners
(I think).
Therefor only scenario I can see where the threshold would come into play
would be if almost none of the SP columns overlapped well with the input
data, and it only took say 4 actively connected synapses to make the top 2%
(instead of 8). In this case the threshold value of 6 would prevent these
not-well-overlapping columns with just 4 active connected syn’s from
becoming winners. Further this could result in not enough winning columns
coming out of the SP, if say the top 2% or columns included 4-synapse ones
but only the top 1% included 6-synapse ones. This would results in just 1%
sparsity instead of 2.
Does this make sense? If so, it seems the threshold is either: 1) having
no effect (as in the first scenario where inhibition would get rid of the
noise anyway); or 2) having too much effect (as in the second scenario
where it would prevent some columns from becoming winners that would be
needed to reach the 2% SP column sparsity).
I wanted to try and answer a question instead of just asking for once so I
hope some of this makes sense. Thanks,
– Sam
1 Like
The parameter is used to help reduce the effects of noise in some cases where the input sparsity can vary a lot. This can happen in situations with topology. Imagine a vision example, where a set of columns are looking at a small portion of the image which happens to be basically blank. The inhibition radius for those columns is local so a small amount of noise in that area would cause columns to become active. We don’t want a small amount of noise to trigger columns - the stimulusThreshold will avoid that.
In the OPF and other scenarios where the encoders are always outputting a reasonable number of bits this threshold is not useful. In the OPF our encoders always output a fixed number of bits. That’s why swarming doesn’t permute it and we set it to 0 for streaming scenarios.
2 Likes
Thanks Subutai, that answers all my questions.
