I don’t have any insights into the official algorithm, but I have also developed my own implementation of semantic folding based on the videos that they posted. The way I have approached this task is in addition to generating the word SDRs, also keep track of their frequency. The frequency can be used to produce “weighted SDRs” (non-binary SDR, where the bits have a weight rather than just a zero or one value).
Two weighted SDRs can be compared to generate a weighted overlap score, which allows you to suggest other words which have a lot of overlap and are frequently used. The top results can be used as “contexts” when doing other SDR math.
One other important point is that the words most frequently used in a language tend to add the least amount of uniqueness to the context. This is a result of Zipfs law. This property is very useful to keep in mind when designing “word math” algorithms, since it means for many tasks you can throw out a large percentage of results and focus in on a smaller subset. I bring this up, because you will find that it is relevant to the “contexts” process I described above.
Wouldn’t the total number of contexts (i.e. bits in the SDR) be the same, or does your hashing function have some scaling property? (EDIT – NM, I get what you mean – you are essentially using random distribution to map points into a smaller space, and relying on property of SDRs that says a lot of random overlap is virtually impossible). Personally I have found that the main advantage to positioning semantically similar snippets close to each other is that you can perform a simple scaling algorithm on the massive original SDRs to produce much smaller working SDRs, so that points close to each other in the large SDR (which are semantically similar to each other) will map to the same point on the smaller SDR.