I’m trying to learn the details of how classifiers work in NuPIC/HTM.Java (are classifiers technically part of HTM theory, since they have no biological equivalent?), but I’ve had some difficulty finding detailed explanations of how they work.
As I understand it, there are currently three types of classifiers:
CLAClassifier
KNNClassifier
SDRClassifier (newest and best?)
I’m not yet sure what the differences between them are, though. I believe that they all serve the same purpose: to take output from the Temporal Memory and use it to predict future values (however many time steps into the future you desire) - and to output these predicted values not as an SDR, but in the actual form that they would be fed into the HTM (e.g. String, float, etc…).
I’d like to gain a better understanding of how all these classifiers work in detail. Are there any good sources to learn about this outside of the actual source code? Most of the sources I’ve been able to uncover lack significant amounts of detail (although, this mailing list thread was pretty good, but it would be nice to have some additional sources and details).
You are right they are all serving the same purpose - to extract useful information from sparse distributed representations. The CLAClassifier used a voting algorithm, where each bit in the SDR vote for a particular class label (based on its activation history). If you are interested in this algorithm, Subutai’s video is an excellent introduction for it.
The SDR Classifier is a simple one-layer feedforward classification neural network. I revised the traditional classification network such that it can learn in an online fashion. Checkout this document if you are interested in the algorithm SDR Classifier. We found that it gives better results than the CLAClassifier. The goal is to replace the CLAClassifier with the SDRClassifier.
Thank you, Matt! I appreciate your posting of that document - I certainly found it interesting. I didn’t know too much about the KNNClassifier prior to reading it. Very good intro to the algorithm! Nice to know a bit more about what @cogmission is working on, as well.
