How boosting changes the SP representation

[breznak]

Currently discussing log boosting and its plausibility and impact on MNIST, and @vpuente shared great biological detail why boosting is used, but probably only in the early stages.

@vpuente

As a note of caution seems like homeostatic synaptic plasticity (the mechanism behind boost) is disabled in L4 quite soon (After the animal was born [1]). Most of the papers about homeostatic synaptic plasticity talk about the inverse proportion between activation frequency and mEPSC voltage [2] (which ultimately affects who will win the inhibition).

In any case, in [2] you can read:

Currently, we know little about the cellular
and molecular mechanisms underlying homeostatic plasticity in vivo.

[1] A. Maffei, S. B. Nelson, and G. G. Turrigiano, “Selective reconfiguration of layer 4 visual cortical circuitry by visual deprivation,” Nat. Neurosci. , vol. 7, no. 12, pp. 1353–1359, 2004.

[2] G. Turrigiano, “Homeostatic synaptic plasticity: Local and global mechanisms for stabilizing neuronal function,” Cold Spring Harb. Perspect. Biol. , vol. 4, no. 1, pp. 1–17, 2012.

Could someone please try to replicate @rhyolight 's results with log boosting? The formula is simple

SpatialPooler with log boosting

@mrcslws

One thing to keep in mind: If there aren’t enough unique inputs, boosting will wreak havoc.

An easy example: my “trivial sequence”: http://mrcslws.com/blocks/2016/03/13/column-overlaps-and-boosting.html

I have 25 unique inputs, 40 active columns, and 2048 columns. That means at most 25 * 40 = 1000 columns will be used. So at any given time, at least 1048 columns are getting starved, and boosting is going to teeter-totter the representations.

@mrcslws Thanks for sharing this, this dates ages back to a problem some NASA guy raised, a SP with boosting on a “dummy” input sequence will produce periodic boosts in activity. Which is unwanted.

Here by “inputs” you mean actual 25 different iptut patterns, not any bits, right?
Basically the issue is with over-provisioned SP.
I’d like to make this into a test.

My takeaway is, I understand what boosting should do and why it helps sometimes, on the other hand I don’t see clearly how it’s done biologically. And we have the edge cases here where boosting hurts.
What do you think of my proposal to remove boosting as is now, and replacce the functionality with synaptic death? Unused synapses are removed (solves over-provisioned issue above), this avoids “weak” columns by simply killing them off. Now we need a mechanism to add new column when all other are busy (well estabilished, connected) and we still lack the precision.