HTM vs. bayesian inference (network), predictive coding

I did a lot of studying about Bayesian inference models and probability theory recently. What Bayesian inference will not tell you about intelligence is HOW it works. You could create a probability-based prediction model for any dynamical system, and it might match the predictions of the system it is modeling, but it does not tell you how the system works.

We are interested in how intelligence works in biology, so we are not focusing on anything Bayesian. Of course there is probability at work within HTM and in your brain. Probability is essentially a part of every process. And Bayesian techniques are extremely powerful. But we don’t think they hold the answer to how intelligence works.

If you really want to pursue these ideas, you should read the 2009 paper from Jeff and Dileep (note the editor of this paper is Friston). Numenta abandoned Bayesian models when Dileep left to found Vicarious after this paper was released (they continue with the Bayesian work towards intelligence). If you read the paper, you can easily see the dichotomous tone between Jeff and Dileep already.

So, as I found out recently, this ground has been trodden before, almost 10 years ago. Jeff and the rest of Numenta are going in the biological direction. Others more interested in probability theory continue onward in the mathematical direction.

6 Likes