# Exploring Reinforcement Learning in HTM

**URL:** <https://discourse.numenta.org/t/exploring-reinforcement-learning-in-htm/1545>\
**Category:** Tangential Theories\
**Created:** [November 1, 2016, 7:30pm UTC](https://discourse.numenta.org/t/exploring-reinforcement-learning-in-htm/1545 "2016-11-01T19:30:34Z")\
**Posts on this page:** 1\
**Showing post:** 8

<div class="post-metadata">

**Author:** ![Paul\_Lamb](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/paul_lamb/32/2725_2.png) [@Paul\_Lamb](https://discourse.numenta.org/u/Paul_Lamb)\
**Post date:** [November 12, 2016, 4:00pm UTC](https://discourse.numenta.org/t/exploring-reinforcement-learning-in-htm/1545/8 "2016-11-12T16:00:15Z")

</div>

> [@sunguralikaan](#):
>
> This limits you on multiple levels. First of you don’t have the ability to produce unknown behaviors.

I addressed this particular problem with the “novelty” concept. It is true that you must be aware ahead of time all possible combinations of motor commands, though (i.e. you couldn’t plug this into any arbitrary system with possible motor commands that are initially unknown).

> [@sunguralikaan](#):
>
> It also implicates that the intelligence with no previous knowledge knows what activities to try.

Definitely this is a core problem with the strategy.

> [@sunguralikaan](#):
>
> every iteration involves multiple isolated stages of stimulating the same region with different inputs. Seemed unlikely at that time and still seems unlikely.

Agree with the “unlikely” observation, and why I have called out this phase of the process as the most biologically infeasible.

[quote=“sunguralikaan, post:7, topic:1545”]  
I may be able to speed you up a couple of months.[/quote]

Despite the obvious limitations, at the moment the “imagining” strategy appears to work for the limited test case that I am writing it for. My development style often involves hooking up components that I know ahead of time to be deficient, and go back to replace them with better solutions. A lot can be learned from building out a whole system in a limited or implausible use case – it gives you insights into other peripheral problems that you weren’t aware of from the outset, and solidifies a higher-level understanding of the overall system. In other words, those couple of months are an important part of the process 🙂

> [@sunguralikaan](#):
>
> I would advise to find a way to stimulate that region with the union of all the motor actions. That is the best theory I have about plausibility and scale that seems to work in practice.

I agree that this seems to be a promising strategy. Where I’ll probably go next when I get to redesigning this phase of the process, is exploring @Bitking 's “boss and advisor” idea. The main deficiency I can see with [my initial thoughts](https://discourse.numenta.org/t/htm-and-negative-reinforcement/1561/6) are the ability for the system to try novel actions, or retry actions that it previously learned to be negative. This will probably involve incorporating the “novelty” concept.

---

_[View the full topic](https://discourse.numenta.org/t/exploring-reinforcement-learning-in-htm/1545)._