I created, tested, and analysed a proof of concept for the Basal Ganglia. The model does not include any sort of motor control so it is not useful but I think it’s an interesting stepping stone. This work is based on the work of (Sungar, 2017). The biggest difference between their model and mine is that mine models the Globus Palidus in detail.
Thank you for reading,
I have some simple questions. I don’t know much about the Basal Ganglia or reinforcement learning, so bear with me.
- Why is it important to have two pathways (stiatums)?
- Is the reward some function of how well the HTM is performing?
- Are you using resets in the TM between sequences during training?
There are some odd interpretation of the two channels; things like go/no-go and such.
Once you place the channels in a larger framework of motor control most of this falls away and it starts to look like the true purpose is something more mundane like extend & retract.
Please see this paper, particularly section 3, with attention to the bit about posture - a mouse balancing against external forces.
Henry H. Yin, How Basal Ganglia Outputs Generate Behavior
I hope this will give you some background to make an informed evaluation of this proposed model.
Then where to the rewards come from?