Yes, I think this is the suspicion. That the concept of grandmother is an SDR i.e. it is very robust and the bits in the SDR are not at all representing some other semantic related to grandmother. I’m supposing that other areas are sensitive to parts of the grandmother SDR (i.e. even individual bits) and that can trigger other SDR representing other concepts like “elderly”. While we can as an observer say that bit X of grandmother correlates with “elderly” because it will excite the “elderly” SDR in another area, within the context of the grandmother SDR, we have a unique SDR that is very robust. But I am just playing around, taking wild swings as I learn more about HTM…
One way to look at this is how you would build an encoder for, say, floating point numbers between 0 and 100. Since you have only a finite number of bits to work with, you cannot give every possible number a unique encoding – there will inevitably be overlapping bits between representations.
Now you could deal with that by generating the encodings randomly (meaning the overlap between any two encodings would be random), but you might instead want to use the overlap to represent semantic similarities between numbers instead. For example, you might want 2.4 to be more semantically similar to (i.e. more overlapping bits with) 2.5 than, say, 8.2. You could use the overlapping bits in a way where numbers closer to each other have more overlap (and are thus more semantically similar) than numbers which are further away from each other.
In this case, a given bit in an encoding would have some small, fuzzy semantic meaning (“in the vicinity of 2.5”), but that one bit by itself is not necessary to recognize the number 2.5.
1 Like
That makes sense in for an encoder. I’m less sure that applies to the output of a SP. Intuitively, I would say the spatial location in an SDR is irrelevant once in the network (it becomes relations that matter).
I’ve always seen the purpose of the SP algorithm being to fix sparsity (and sometimes to change the scale up or down) while preserving the ratio of semantics.
True. A property of SDRs is that overlapping bits do not have to be physically close to each other… they just need to overlap. However, there is an interesting relationship between topology and semantics (through hexagonal grids) that I have hinted at a few times on the forum, so just because the SP algorithm shuffled things around, one could reassemble the activity back into topologically relevant encondings purely from the encoded semantics themselves.
3 Likes