Numenta has paper on math properties of SDR.
Because I’m not mathematician, I’m asking if somebody can speculate on the properties of Segmented SDR.
SSDR is my ‘invention’. Very simply it is sparse vector like SDR with the added constraint that every bit is selected from a segment, rather than ‘totally’ randomly.
Let say n=1000 and w=50, then we divide them and segment_size=1000/50 = 20
So we divide the vector to 50 segments of size 20bits.
Then we pick randomly one bit from every segment.
How would union,partial match, overlap, noise-robustness and other properties fare.
In general I understand those are preserved more or less. I just want to know more precisely.
Second what if the selection in a segment is not random ?
Why … this way you can combine localist representation (selection within segment) with sparse (% of n).
2 Likes
It seems like you are doing Jittered Sampling (at least that’s how it’s called in Computer Graphics). It’s used for faster de-noising when a lot of sample is needed among other things.
Basically you are trading a portion of the randomness for even distribution. Random SSDRs will have a higher overlap score and easier partial match. Letting two SSDR to accidentally match each other easier (tho since SDR’s capacity grows at a super-exponential rate, it should be fine unless the segments are too small). But it would be better at communicating the distance between representations since the density is very stable and very few bits can overlap by random chance. And locality/topology makes sense in this configuration.
Edit: A bit of experimenting using your parameters versifies my speculation.
2 Likes
what are the X and Y axis … X: overlap, y:n
Exactly. I generated 100K SDR and plot the distribution of overlap
Hi raptor,
That analysis, I mean SSDR, already exists.
Cheers,
I have somewhere a maths paper from the, let’s say, “Pentti Kanerva circle”.
Sorry, I have to find it, that seems a hard task.
I’ve read almost anything from Kanerva … did I missed some ;(
High-Dimensional Computing with Sparse Vectors
This SDR model does this: http://www.sparsey.com/
1 Like
