Reward hacking in simple HTM agents (using OpenAI Gym)

[Bitking]

In the subcortex there are multiple nodes, each evaluating current needs and asserting a goal state based on which one screams the loudest. For example thirst and hunger both drive action but you might ignore food if you are really thirsty. After the thirst is satisfied the hunger wins and you turn to finding food.
Maslow describe a pyramid of needs and as you satisfy the lower needs the higher level goals are able to be activated.In the post on processing in the old brain, I build to a global goal state, driven by inputs from local sub-processors.

Low level processing in the old brain

I have posted several times offering that the lower-level control structures work something like a Boltzmann or Hopfield network. I would like to expand on this concept. Let’s start with the basics. In the brainstem processing, we have many body sensors that sample blood-based messengers such as osmotic concentration or glucose levels. All of the body sensations also pass through these systems and provide additional information on the outside environment. It is the job of these low-level contr…