Looking through the nupic base code synapse sample size parameter is set to 20 but the max synapse per segment parameter is 255. Moreover, numActivePotentialSynapses counts however many synapses on a segment that has permanence >= 0. This seems to suggest to me that any given segment maxes out at 20 synapses which is enough to track SDRs fairly reliable. My question is why even have a max synapse per segment parameter at all? It seems like the newSynapseCount calculation enforces the segment to max out at 20 so what is the point of allocating space to synapses that won’t ever materialize?
Hi, Synapses can be added to the segment every time the segment learns. Each time it learns it can form at most 20 new synapses. So over time it can accumulate more than 20 synapses.
Thank you dmac. I follow what you are saying and that’s how I take the theory to work as well. However, in the code itself, if there is any permanence (i.e. synapse) on a segment the counter for numActivePotential increases for each one. As a result, as soon as a segment has 20 potential synapses it will cap out. This is because newSynapseCount = synapseSampleSize - numActivePotential and synapseSampleSize is = 20.
Is step 2) Activate a set of dendrite segments
supposed to be done for every segment in every column? Perhaps, this is where I’m going wrong bc I was under the impression this was only performed for predicted cells.
if there is any permanence (i.e. synapse) on a segment and the presynaptic cell is active then the counter for numActivePotential increases for each one
So a segment may have a total of say 200 potential synapses, but if only 8 were active at t-1 then 12 new synapses would be grown on that segment (20-8). Right @dmac? So the idea is to:
set a ceiling on total potential synapses per segment (255)
set a ceiling on number of synapses grown at once (20)
grow fewer new synapses on segments that were more predicted (the 8 numActivePotential from t-1 gets subtracted from 20)
Thank you for that clarification! I was overlooking that additional criteria and now that makes sense.
One follow up I have is what happens once maxSynapses on a segment are reached. Will the synapses simply decay (due to permanence decrement) so just wait until they fade away before adding new ones? Or is it better just to grow a new segment?
Right, the decrement can eliminate potential synapses by bringing their permanence values back down to 0.
This decision is part of the TM logic. When a column bursts, a search is done for the bestMatchingSegment of any cell in that column. We know that no segment was ‘active’ - since that would’ve made its cell predictive and prevented the column from bursting - but the column could still have a bestMatchingSegment. To qualify as ‘matching’, a segment must reach a certain threshold in number of active synapses - meaning that it was correctly predicted enough to ‘match’.
If the column has a bestMatchingSegment the column’s winnerCell is the one holding that segment. If the column does not have a bestMatchingSegment the column’s winnerCell is chosen randomly.
