Replace inhibition with proximal horizontal inhibitory connections

Science Numenta Theory
1

The mechanism for Inhibition is well known and accepted.

The implementation uses (topological) neighbourhood in which cells compete for activation (k-winner-takes-all).
Now, the problem is with biologically defining the neighbourhood.

I propose to add proximal horizontal (to peer cells) connections within Spatial pooler.

  • peer connections randomly sample from distribution, same as feed-forward conns.
  • a “negative” Hebbian rule applies: each time 2 cells fire (activate) together, their peer connections strenghtens (but the connection means how much/easily we’ll inhibit).
    • if only 1 cell of the pair activates, the connection decays.
  • each time a cell is active, it sends an inhibitory signal (weighted by strenght of the connection) to its peers
  • each time cell’s (negative points, accumulated inhibition) decays.

This way, we can simplify the process of inhibition and its neighbourhood/radius computation.

Extension:

  • special “types of neurons” can have different proximal connections structure, and thus define arbitrary topologies.
4 Likes
2

Hi breznak,
I don’t understand your point. Are you saying that the winner take all inhibition of the spatial pooler needs topology to be biologically accurate? If so… we tested/worked on this idea and the following might be useful.

Biologically, the inhibition of the spatial pooler is likely to come from the ubiquitous fast acting basket cells. They have a radius of innervation. After testing the spatial pooler with a winner take all approach we wanted to verify that we would get similar results using local inhibition. We tried a few different algorithms for this and found that it worked. It is simpler to do a k winner take all so that is what we stuck to.
Jeff

4 Likes