Why is the typical size of an SDR 2048 and the typical number of active (or "on") bits 40?

Science Numenta Theory
1

In the BAMI book, under the section Capacity of SDRs and the Probability of Mismatches, it’s written

With typical values such as n = 2048 and w = 40

It isn’t the first time that I see these two numbers as suggestions of typical values for these parameters of an SDR. Is there some theoretical or practical/empirical reason behind this choice of typical values? If yes, what is it? (If not, don’t you think that it is a little bit unscientific to suggest “typical” values that have no “scientific” or “experimental” support?)

In that same section, it is stated that SDRs have a smaller capacity than a dense representation, but, for all practical purposes, if the choice of n and w is sensible, then we can use SDRs to represent an astronomically large number of different objects. Is this the only reason why people say that n = 2048 and w = 40 are the typical values?

In later sections, topics such as “inexact matching” and “subsampling” are described in the context of SDRs. Apparently from the specific mentioned examples, these numbers n = 2048 and w = 40 are, at least in terms of false positives (i.e. matches of two SDRs that are considered equal but are not), (more or less) “appropriate” when using “inexact matching” and “subsampling”, as the probability of false positives with n = 2048 and w = 40 is relatively low.

2

We want to represent things in the same way the brain does, and we know that in the neocortex, about 2% of your neurons are active at any point in time. Knowing the representational capacity of SDRs is 2%, we know that 2048 bits in an SDR is more than enough to represent any value we like until the end of time, so there is not much use making it much bigger. So we keep the sparsity at 2%, which happens to be 40 on bits.

This size representation has an enormous represenatation capacity, and it makes sense when you try to relate it to the neuroscience because each bit in the SP SDR represents a group of pyramidal neurons, each sharing receptive properties over the same input space.

2 Likes