Isnt CLA classifier just like I gather a bunch of inputs (sequence)
for example A B C D E F and A B C D J K
I calculate the frequency of them, then I can know when D happen, I will get like 70% (maybe) chance of E and 30% chance of J? Why we need to do the spatial pooler and temporal memory anyway?
How does temporal memory work? How can it learn? I read the white paper of temporal memory algorithm and it’s saying that the segment or synapses are continous learning, but how that can related with the lookup table for the CLA classifier?
The benefit of SP is that you enforce sparsity, which comes with noise and error tolerance. A couple bits being off will have virtually no impact on your result. The benefit of TM is that you get D in context for this sequence (D after C after B after A). Thus you don’t have ambiguity with a D that is in some other context like Z Q R D X Y for example. See single order vs high order memory.
I’ll let others comment on your question about how TM relates to the CLA classifier, as I’ve not really had any experience with the latter myself.
I understand that TM is to let the system know the context of the input but it still not necessary for the CLA classifier.
For example, I want to predict two steps ahead, this means that if there are two patterns A B C D E F and Z Q R D X Y, I just histogram these two pattern and when I reach D, I can know that maybe the likelihood of F happen next and Y happen next. Still, I dont need HTM theory, do I?
What I can only imagine why this classifier need CLA is about the performance but when I know that it takes lookup table for every bit I think it still needs a lot of storage for data.
Another point that I think CLA is useful for this kind of classifier is that, it can gather a sequence of input into one object and let it become a pattern but still a histogram can also do that.
Am I missing anything important here? I just want to fully understand why I need this classifier for CLA and what I should understand for developing CLA and maybe implement something else based on this theory
This sounds like a fixed order Markov Chain, which you could use instead of HTM. It’ll require that you set window sizes – for how many steps in the past to use for prediction, and how many steps in the future to predict. Any such system (IMO) will often end up 1) using too much or too little context for prediction, and 2) predicting either more or less than it’s equipped to – when applied to real data.
The TM overcomes this need for windowing by using a hebbian sequence-learning mechanism, based on forming and adapting distal dendrite segments in the neocortex. This allows it to theoretically make any number of predictions, based on any amount of context at any given time.
As for the Classifier, its purpose is to map the TM’s predicted states (which are sparse binary arrays) back to the raw data type. So the Classifier does not make predictions itself, but rather classifies the predictions from TM within the raw data distribution. So the input TM predicted state could be like 00001000001101000011000… and the output could be like {‘A’:0.7, ‘B’:0.2} or {‘1.37’:0.9, ‘2.81’:0.1}.
If you want to know how the Classifier works there’s an informative video. The original version was called the CLA Classifier, but the latest version is called the SDR Classifier:
