I came up with an structure made of HTM neurons that maybe has the ability to learn with realtime data ,rewards and intrinsic rewards (partly inspired by
Proposing a Model for the Basal Ganglia and Reinforcement Learning in HTM )
and want to code it out to test my ideas.
The environment I choose for my agent to learn and play is OpenAI Gym’s MsPacman v0
There are two types of input(observation) I can choose from: Ram and Game Image
For Game Image, Gym will return a 3D (210,160,3) array (210 and 160 are the height and width of a emulated Atari screen while 3 means R,G,B).
For Ram, Gym will return a 1d array with 128 numbers from 0 to 255, derived from the emulated Atari machine’s ram.
From this forum I learn that encoding vision is a very hard problem which you haven’t solved completely.So I have to choose Ram.So how should i encode the ram properly?
Ram is something like this:
(here shows Ram in consecutive 3 timesteps)
For me, i can only think of scalarEncoding the numbers in the ram one by one and stack them to make a new SDR
scalar encoder whose resolution = 1, w = 21 , minval = 0 ,maxval = 255 encodes 128:
I have only gone through the HTM school videos and read some example code.I understand the meaning of the parameters, but don’t master the ideas of encoding as well as […is this grammatically correct…ah…im not a native speaker, plz don’t misunderstand this!] most of you in this forum.So I am asking : how should i encode the ram properly so as not to lose the information in the ram (and also…effectively)
By the way, I know MultiEncoder can be used to make different fields of input into one SDR…are 128 fields too many?
And, if field Alpha uses 100 bits to encode while field Beta uses 10, will Alpha become more “important” than B?Should I make every bit encoding B 10 bits, or the network will figure out that Alpha and Beta are of the same importance?