An open-source community research project on comparing HTM-RL to conventional RL


#62

Yes, a multiple-field “classifier” is what is needed.
The only info i see on the github issue is a statement of the issue and several messages indicating no progress on including it in Nupic. Is there some other thread or comment that suggests how to implement it within the Nupic framework?


#63

The only caveat I would add to this particular classifier strategy (there are better classifiers that NuPIC has implemented) is that the percentage of SDR overlap alone isn’t always sufficient to determine a likelyhood score of next predicted state.

Consider, for example you have well trained the following sequence:

ABCDEFG

And consider you have also encountered the following sequence much less frequently, but enough that it has grown “max new synapses” for each step:

ABCDXYZ

Then you input ABCD. At this point, both E and X would be predicted. Both would be equally represented, so you might think there is a 50% chance that either could happen next. With a better classifier, though, you could determine that E is far more likely.


#64

I should clarify that this issue is just to add the functionality to the OPF. You could do it yourself with the Network API as-is. Sorry if I confused you @Randy @matan_tsuberi

Perhaps some relevant docs, since you are talking about NuPIC interfaces: http://nupic.docs.numenta.org/stable/api/index.html


#65

So I think we should use the aforementioned approach.
Anyone objects? Any improvements? Any variations on this approach?