Free Energy Principle



Does anyone have any experience thinking about the ideas in Karl Friston’s Free Energy Principle and how they relate to HTM? Seems like many of the same ideas, especially when it comes to prediction and action.

White Paper: A Model for General Intelligence

I watched to get a feel for the principle.

Seems as though they’re delineating the relationship (the feedback loop) between prediction and attention; that the brain chooses what to attend to in order to maximize its ability to predict in relation to its currently developed models (identity).


While I haven’t personally thought about it much myself, it reminded me of this thread which seems to be touching on a similar possible driving force.

From the perspective of someone who was just introduced to the theory, on the surface it seems fairly self-evident. I could see it being useful when weighing new theories or ideas – how well do those ideas align with the goal of reducing entropy (ultimately meaning better predictions). On the other hand, that kind of tends to already be the goal of new theories for AI anyway…


At a high level, it feels right, but what is up with this? I took this from Karl Friston’s slide deck from a 2016 talk.

I don’t think this represents current neuroscience ideas. Are each of those blue boxes supposed to be Bayesian models?


I have no idea about the Bayes question but the localization of functions is way off.


This paper, “The free energy principle for action and perception: A mathematical review”, may be of help for at least coming to grips with the math of FEP:

Also, “The Predictive Mind” by Hohwy offers an account of Friston’s work at more of a layperson’s level; Friston himself corresponded with Hohwy during the writing of the book and provided feedback to the author.
This may be a good place to start if you find Friston’s papers to be a bit overwhelming (like I do).


And way over here there’s me saying that the brain is a motor pattern learning and selection machine.
The body has needs and picks the best motor pattern(s) to solve those needs.

Consider this from Patricia Churchland in her book “Neurophilosophy”:
“If you root yourself to the ground, you can afford to be stupid. But if you move, you must have mechanisms for moving, and mechanisms to ensure that the movement is not utterly arbitrary and independent of what is going on outside. Consider a simple protochordate, the sea squirt. The newborn must swim about and feed itself until it finds a suitable niche, at which time it backs in and attaches itself permanently. Once attached, the sea squirt’s mechanisms for movement become excess baggage, and it wisely supplements its diet by feasting on its smartest parts.”


I found this blog article recently that does a good job at explaining things:


Free energy? Where have I seen this before?

Points to this paper.

Surprise and Disambiguation in L6