I’ve been thinking about how to implement a MIDI encoder, and I re-watched the HTM School video’s on SDRs and encoders. One thing I’m confused about is how those come together.
If an SDR has a typical sparsity of 2%, and an encoder needs certain overlap between values to represent semantic similarity between the values, does that not contradict in a way? How should I practically combine low sparsity with high overlap?
To make it a little more concrete, lets consider the datetime encoder shown in Episode 6. I know it’s an example to make the principles clear, but it has an n of 328 and a w of 48. That makes for a sparsity of 14,6%. Would it be enough to increase n to 2048 without changing w? (sparcity = 2,3% but close enough). The overlap between values cannot be changed, so would that not cause other problems, like influence the risk for false positives, or the noise tolerance?
Edit: misspelled “sparsity”.