Hi,
Please forgive me asking so many questions.
If I only use the TM, how can I keep the input SDR a specified sparsity without the SP learning process?
I am going to feed the massive category data (more than 1000) to the TM directly, is it possible?
An easy and cheap way would be to randomly drop out as many bits as you want. There will still be semantic information in the SDR.
The encoding vectors have fixed sparsity. They usually go into the SP to produce a sparser vector, but they can be fed directly into TM.
When using categorical data the TM relies on some of those categories repeating, since there’s no semantic overlap between categories. So if the TM is seeing novel inputs all the time it won’t be generating a lot of predictions – mostly just bursting columns. I’d say try and reduce the number of distinct categories as much as possible – maybe grouping them into larger categories so there’ll be more periodicity in the inputs.
2 Likes