Spatial Pooler Implementation for MNIST Dataset

Too many questions :slight_smile: (for my poor knowledge)

Homeostatic plasticity seems to be a really important thing. Is critical during embryonic cortex development and early stages of life. My hypothesis is that when the inputs stars to come in, it balances mini-column distal synaptic load across the cortical column. Once the animal has acquired “the base” knowledge, homeostasis is progressively fade away because it will do more harm than good in L4.

My hypothesis is that L4/SP at birth is barely connected (with a large potentialPCT).

Unfortunately, if you instead of using a 0.5 prob of being connected in SP, uses a 0.01, you will see that every input lands into very similar output value (if PCT=1, all in one)

That necessary initially connected synapses might have an impact on system evolution: it the random is not aligned with the input stream it could prevent a homogeneous number of connected synapses per mini-column in the TM. Besides, that 0.5 is not good for noise tolerance.

I think a strong boost is necessary during early learning. If you use a barely connected SP. Once the number of distal synapses per mini-column is balanced, disabling it progressively seems the right thing to do. My intuition is strong initially SP will perform a really strong clustering and boost will “split” the fine detail inside the cluster.

I understand synaptic competition but looks not very bio-plausible (At least, I couldn’t find any evidence of it). Heterosynaptic plasticity [1] does something like that but this is already in the learning algorithm (is the forget of non active synapses).

[1] W. C. Oh, L. K. Parajuli, and K. Zito, “Heterosynaptic structural plasticity on local dendritic segments of hippocampal CA1 neurons,” Cell Rep. , vol. 10, no. 2, pp. 162–169, 2015.

1 Like