Hi everyone, I’m looking for some guidance regarding all the different knobs you can tweak in an HTM. I’ve seen in my own experiments that even relatively small changes to the number of cells, the permanence increments, the boosting strength etc. can have significant effects. So I’m wondering if anyone could share what intuitions (if any) they’ve learned on this matter. Is there a good rule of thumb when choosing all the parameters? Or do you just go with the defaults and then use trial and error to optimize them?
I’ve found it most impactful to optimize the encoder settings rather than the learning parameters. I think the default hyper-parameters have been shown to work well across many data sets, though if I were to change anything it’d be the permanence incr/decr ratio and maybe connected permanence.
As you probably know this sets the rate of learning vs forgetting, as well as how many times a pattern must repeat before the synapse becomes permanent. So if you want 1-shot learning for instance, the perm incr should equal connected perm I believe, so the perms would jump from 0 right to the connected perm value.
I’ve also found it helpful when dealing with small data to turn off spatial pooling. This can help the TM to stabilize faster, without SP learning going on.
Also if you’re doing a forecasting application rather than anomaly detection you could try swarming, where you choose a predicted field and it finds optimal param values for accuracy on that field.
There are a lot of interesting things to know about the parameters and how they work. You can learn about them through experimentation, and maybe you will get an intuitive understanding.
To find the best parameters, you need a parameter search, like grid search, evolution, or swarming. There are too many parameters in an HTM to optimize by hand.