No, I’m saying a deep network can be trained online, just like HTM is. See Deep Q-Networks for example. The form of experience replay they use may be distasteful if you want biological plausibility, but the fact that it successfully learns online is beyond doubt.
There’s a machine learning paradigm called “unsupervised learning”, in which you don’t need labels on your training data. Autoencoders, generative adversarial networks, self-supervised learning in the form of prediction. These are all ways to do machine learning without labelled training data. And then there’s reinforcement learning, as Charles points out.
A lot of people have demonstrated transfer learning in deep networks already, including myself, in which you use a network that was trained for a particular task, i.e. ImageNet classification, and use it for a new task, like robot navigation, with or without fine-tuning the weights.
And you seem to be suggesting that HTM is somehow better at this? The brain is better at this, sure. HTM in its current state absolutely does not in any way improve over the state of the art in transfer learning. This would be a big deal and there would be Nature papers to read if it did.
See my comments above. These properties are far from unique to HTM. And you greatly overstate how well HTM, in its current state, can actually do them.
Of course. It’s a work in progress. But a developing technology has never been done any favors by overstating its capabilities. That’s how you overhype things and alienate people who take very seriously the kind of claims you’re making.
There is no guarantee that DL can “keep pace” with HTMs, but there is also no guarantee that HTM is a correct theory of the neocortex. Furthermore there’s a reasonable possibility that, by ignoring the constraints of biology, machine learning will far outpace bio-inspired solutions. It would be an act of pure unjustified faith to claim otherwise. Obviously I think a lot of inspiration can be found in the brain. But it’s unquestionably not the only way to solve intelligence, because there exist an infinite number of equivalent algorithms to solve any particular problem (proof left as an exercise for the reader).
I’ve heard from the HTM community a lot of criticism of machine learning, and of deep learning in particular. I’m all for scientific criticism, but unfortunately the specific criticism tends to betray a shallow understanding of the field and the technology. I do recommend going out and implementing solutions to real problems to evaluate the merits of these different technologies. The hype over any particular idea can safely be ignored in favor of real performance on real problems.
After all this I feel the need to reiterate my commitment to HTM. I think forming sparse connections on independently plastic connection sites to sparse population activity patterns is going to be the key. So HTM is going in the right direction. But claiming anything more grandiose at this time is just marketing, and therefore can and should be ignored.