I think there’s actually a lot of overlap between how HTM and DL work, at least if you just look at the spatial pooler.
Compare this explanation of deep learning to this explanation of SDRs. In both cases, neurons are partitioning a possibility space and labeling a particular subspace as the space of recognized patterns. The difference is that DL uses linear algebra and topology while HTM uses combinatorics and binary vectors.
Then, take into account the fact that DL is starting to use less and less precision for synapses (DL has been one of the major reasons that AMD and nVidia are supporting half-precision floating point, and I think Google’s TPUs might even be using 8-bit values), and the fact that modern DL often uses sparse vectors, and I think it goes to show that HTM and DL have quite a bit in common.
The differences now are that HTM takes low-precision synapses to an extreme, and that HTM has temporal pooling and feedback while DL does not. Sure, DL has backprop, but that’s just what you need to do its topological transforms efficiently. I’m quite sure that you could swap out the learning algorithm for something else if you changed the way it partitions the input space, and still wind up with a very efficient learning algorithm.
That said, there is another big difference between HTM and DL; reinforcement learning. Currently, if you want to train a DL network to do a useful task, it’s very easy. If you want to train HTM to do the same thing, you have very few options. I think what HTM needs is some form of reinforcement learning. There’s plenty of evidence that it occurs in the brain; we just need to find out how it works. I’ve suggested a potential starting place before, so if anyone wants to try to implement reinforcement learning in HTM, while this may not be a perfectly biologically accurate method for reinforcement learning in HTM, but I think it would be a good place to start.
I would be working on this myself, but I’m too busy right now. I’ve read a lot about HTM, and made a few small toy implementations in the past, but I haven’t had time to get into NuPIC at all, so I’m not sure what it would take to do this with the current framework. If no one else is willing to try, I’ll probably get around to it in a few months. My main side project right now is an experimental compiler, so I’ll need some code to test it on anyway.
TL;DR: HTM and DL are very similar when you ignore the temporal pooler, and are mostly solving the same kind of classification problem. The main advantage DL has is reinforcement learning, making it very easy to train DL networks to do useful things. I’ve suggested a biologically-plausible way to implement this in HTM before, but lack the time to try it now. If anyone else wants, feel free to try to implement it, as it’s probably a good place to start.