I found another method to control the synapses. Normally the synapses in the spatial pooler have a strength of 1. Instead, divide by the strength of each synapse by the total number of connected synapses to the postsynaptic cell.
So, instead of comparing cells based on the total number of synaptic inputs, this compares cells based on the fraction of their synapses that are active. The range of inputs to all cells is now in the range [0, 1]. Cells can compete to activate on a level playing field even if they have different numbers of connected synapses, whereas normally cells with more synapses will have an advantage over cells with fewer synapses.
This is a technique that I’ve seen done by other similar NN models.
Results:
I re-ran the previous experiment, using the MNIST dataset, and I used numenta’s exponential boosting algorithm. It achieved 95% accuracy, as expected.
This method does not directly control the number of connected synapses, and as you can see from this histogram the spatial pooler still has cells which are connected to every possible input as well as cells with very few synapses.
Despite the wide range in the number of connected inputs, almost all of the cells are equally utilized. Almost no cells are stuck off, and no cells are stuck on.
And finally, the entropy of the cellular activity is 99% of the theoretic maximum!