In the HTM school videos Matt talks about encoders. In this overview, he lists a number of properties that an SDR created by an encoder should have. One of those properties is that a semantically similar input value should produce a similar output SDR and that a dissimilar input value should produce a dissimilar output SDR.
I don’t understand the reason for requiring semantic similarity - wouldn’t spatial pooling learning happen just as reliably on randomly (but deterministically) generated SDRs?
Oh, I maybe just realized, is it because of the union operation that allows you to match against an incoming input value more quickly than scanning through all the SDR representations that have been seen?
I think, (but I’ve been wrong more often than not) that what this means is that the first time an encoder encounters an input pattern, the SDR is generated randomly. But when later the same (or even very similar) patterns are encountered, they should produce the same SDR.
I think it has to do with these three learning rules @subutai is talking about here. The second one generates random connections, but the first and third rule encode for the same output SDR on a dendritic segment and improve the reliability for future firings.
Possibly the best example of how semantic similarity is useful is to look at a real world system that uses SDRs. The good people at Cortical IO use SDRs to encode words where they end up with semantic meaning encoded in the bits selected. After this they can perform useful sorting and selection operations.
Please check out these videos for a rundown of what they are doing:
@Falco, yep, I understand that the same input should output the same output, otherwise you could never learn anything. My question is why two inputs that are similar should generate similar (but not identical) SDRs.
I think this is true for categorical data where the inputs have no semantic similarity. The semantic similarity for other data types like scalars allows the TM to generalize better. For instance if it had seen the pattern ‘5, 2, 8, 5, 2, 8, 5, 2, 8’ and then got the input ‘5.1’. Although it has never seen that value before, its encoding vector must have a lot of overlap with that of 5.0, which allows it treat 5.1 similar to 5 and make many of the same predictions.