How keep 2% active bits SDR wihout using inhibition


#1

Hi everyone,

Can i ask you about spatial pooler?
how can keep 2% active bits if I donot use inhibition phase?
how boost factor effect to 2% active bits?

Thank.


#2

Without global/local inhibition. The constant density won’t be kept.
And (with inhibition on), boosting does not effect the result density.


#3

Thank you.
I read this post: Thoughts about topology
as i understand, with no topology, it means no calculate neighboor? if so, cannot keep sparity = 2%. is it right?
please correct me?


#4

Without topology. Cells are not limited to seeing neighboring inputs. Instead they see all of the input space (besides the connection that are not made when initlizaion). In this case, global/inhibition will keep density at a filed value.


#5

I am sorry. I may dont understand clearly.
After calculate overlap between input and column, calculate the inhibition by group some columns and then find max value of overlap.
Can you say more clear for me? Thank you so much.


#6

You get everything right!
A SpatialPooler operates as the follow:

  1. Calculate the overlap score
  2. Apply the score based on duty cycle and boosting factor
  3. Calculate inhibition (find columns with the highest overlap score)
  4. Update connection weight if learning is enabled
  5. Update active duty cycle for boosting in the next step

#7

Thank.
With topology, we choose a part of input as proximal synapses. it means that inhibition in the input.
And without topology means, we choose all pixels in the input to make connections to one column.
please correct me?


#8

Okay. I get what you are feeling confusing about.

Topology means that columns are only connected to a local area of the input. Its like Conv2D in deep learning. Each output is only connected to a sub part of the input image/
For example in the image. Each cell in L2 is only partially connected to L1. With out topology, cells in L2 will connect to all cells in L1.

Inhibition means inhibiting some active columns that has a weaker overlap score being acquittal active… For example setting activationThreshold = 10 and set SpatialPooler to have a 50% density (50% is way too high in realistic situations. Only for illustrative). And your overlap score of each column is [8, 11, 16, 15, 12, 16]. Since the activation threshold is 10. the first column will not be activated. Therefor the score become [0, 11, 16, 15, 12 16].
If inhibition is not enabled. These are your results. All active columns will output an 1. Making [0, 1, 1, 1, 1, 1] the output. But with inhibition. The algorithm looks for the top most overlapped columns and only makes them active. So we are looking for the top 3 columns in this case. That makes the score [0, 0, 16, 15, 0, 16] and thus the final output being [0, 0, 1, 1, 0, 1]. This insures SpatialPooler outputs a SDR with somewhat consistent density.


#9

Thank you for your explaining.
Can I ask more?
how affect if i disable boost factor ?


#10

What boosting does is it (almost) randomly gives columns with some chance being active to actually be active; encouraging broader learning. If you disable boosting after training. The output SDR will become less noisy and might be a good thing.

But if you disable boosting from the beginning. You limit the chance of some columns to learn. Making the SpatialPooler less effective.


#11

Have you seen these videos? There is an episode that explains boosting.