x1 x2 x3 x4 x5
y1 y2 y3 y4 y5
z1 z2 z3 z4 z5
o o o o o { o, o, o, o, o } --> places where inputs are connect to the columns (called columns)
Imagine these are 5 columns each with cells: x, y z (…not seen in video visualizations)
Imagne that each “o” (connection) is what is seen in the video as a column
Maybe @rhyolight can explain it better, but I though that might help…?
No, it’s my long way of being more specific than the options you gave me.
Answer 1. --> No the term was used correctly
Answer 2. --> Columns can have a topology (i.e. can be contained in a multi-dimensional array) but the visualizations don’t specify that.
Answer 3. --> Not a 3D matrix, but yes the “dots” can be thought of as having cells “behind” them.
The answer I gave is correct. The dot’s in the visualizations represent columns and you are correct - cells are not shown.
So, the closest answer is 3 - but I just wanted to make the distinction of there not being a “matrix”, for accuracy’s sake…
What is the point of having the connection threshold as a variable if the spatial pooler simply aligns the permanence values in a normal distribution around that threshold? Why not simply set it to 0.5?
I believe it is so that the initialization behavior is to more quickly bring columns to a connected state with fewer cycles during startup, after which the distribution should vary more widely after the SP settles (as columns increasingly polarize toward the values they have affinity to). Please correct me if I’m wrong - but I believe this is the reason.
I don’t think @cogmission actually answered your question, but I know what you are getting at. Since the connections are automatically distributed around the connection threshold, it doesn’t seem to matter if the threshold is 0.3 or 0.8 (for example). There will still be 50% initial connections and the behavior doesn’t seem like it would change much. The only difference is that the connection activity is happening at different distances from the permanence limits (0.0, 1.0).
I don’t know the answer. But @alavin probably does.
Thanks for bringing this to my attention @rhyolight. SP learning is the answer. The distribution about the connection threshold is just an initialization. As the SP is exposed to more and more data input, it’ll modify the synapse permanences to best detect features in the inputs. So if the threshold is 0.8 (following your example), you can see many permanences increased to the max 1.0. This has several effects, including limiting how “permanent” a synapse can be, which means it’s more at risk of dropping down below the threshold and becoming disconnected. I hope this helps!
Hi! These videos are great Matt and friends!
Now, is the sub-sampling you refer to, the fact that a column only connects to a fraction of the input space? or an actual sub-sampled vector that is passed to each column from the current input pattern?
Its’ that each column only connects to a fraction of the input space. Sub-sampling means that each of the 2048 columns samples a subset of the input, which is the encoding bit array of the raw data. Each column gets an overlap score, which is the number of encoder bits its connected to that are currently active. The columns with the highest overlap score (top 2% I believe) are chosen as active columns. This set of columns is the output of the spatial pooler which is passed into temporal memory.
Oh wait… I did! 