HTM constants confusion

As a general answer to your post:
For determining the default values for the hyperparameters, I always just copy off of The HTM Cheat Sheet.

Some answers that are a bit more specific:

You could set them as the same values, but it wouldn’t hurt to have them as separate variables just in case. :wink:

I don’t think you meant this, but still: there is no (random) initial synapses/segments in TM.
If you obviously didn’t mean that: when a new distal synapse grows from a dendritic segment, you would want it to have a very close to zero permanence, but it depends of course. :grin:
The reasoning is that it could’ve been a noise or a very short trend that might never happen again in the future.
An initial permanence of something close to zero requires few extra encounters of the same (temporal) pattern to happen, so it makes sure that the pattern was indeed not a noise and is persistent.
But if you want your system to be more responsive and catch up with the patterns that come and go rather rapidly (which isn’t ideal for HTM, I don’t think.), set it as a high value of something close to the threshold.

The hyperparameters depend a lot on what you want to do with what data. :slightly_smiling_face:

Yes, it doesn’t grow any new synapses if the number of the active synapses were the same as/exceeds SYNAPSE_SAMPLE_SIZE, but it doesn’t directly affect the maximum number of synapses of a segment as the segment could also be looking for one or two more distinct (spatial) patterns.
No, it doesn’t affect learning much because of extreme sparsity. This is why HTM does sub-sampling(having the relevant synapses just up to SYNAPSE_SAMPLE_SIZE), because the sub-samples are sufficient and robust for detecting the input(context, in this case) the segment is looking for.
Sparsity makes sure that two patterns don’t overlap in ways the system can’t tell them apart. And this allows sub-sampling to be robust enough to be used in practice. With less cost of memory and computation, you can safely assume it’s referencing a unique pattern.

1 Like