HTM Based Autonomous Agent

Science Related Papers
23

Hello @sunguralikaan,
First of all congratulations for this amazing thesis work! I am really impressed by it.

I have some detail questions that I could not quite get from the paper (maybe overlooked) regarding the L5 layer where everything integrates. As I am currently playing around with RL and HTM using a customized Nupics network API orienting on your architecture as the experimenting and researching you did is extremely valuable to learn from.

In you model the cyclic flow is described the following:

  • L5_SP layer is pooling from the feedforward input from L4 (spatial, weighted = active and predicted neurons as the TP in Nupic supports, but no temporal abstraction)
  • L5_TM is predicting the active columns in the context of L2 and L4 active neural input.
  • The active columns of the L5_SP layer are feed forward input for D1/D2
  • The active neurons of the L5_TM layer are distal input for D1/D2, which both try to predict the next state of L5 with the given information. This predictions are utilized to determine the TD error.
  • L5_Apical connections to D1 and D2 are activated and permanences updated depending on the TD error.

In L5 we now have active cells, distal depolarized cells and apical (D1/D2) depolarized cells.

  • In the given framework it is stated that cells which are depolarized by distal AND apical connections will become active.
  • The next step is learning the motor-activation by association. This is done through apical connections from the L5 layer to the motor neurons. Here learning depends on the firing-type leading to either excitation (D1 activated) or inhibition (D2 activated).

In the last integration is where I am slightly confused.

  1. Does that mean Motor neurons only connect to neurons in L5 that are “voluntary” active - meaning active through a combination of apical and distal depolarization? Or also to the neurons that are active through proximal input interpreted in distal context? How do this neurons then influence the excitation level?

  2. Are the apical connections (formed at previous timesteps) from D1/D2 also used in L5 to sparsify the active columns to neural activation (additionally to using distal connections to L2/L4)?

In case of 2. I believe not. But if not wouldn’t it be possible to input (and learn) apical D1/D2 and L5 distal connections directly to the Motor layer and then take their intersection to excite/inhibit the neurons without loss of generality from the algorithm (as the learned apical connections are only used for Motor excitement?)?

This would be interesting, as NUPICs Network API does not support symmetric computation well and it would need a lot of customization hacking.

~

Maybe you could clarify some of it and I hope you support the idea of a NUPIC implementation that builds on your architecture.

@Gary_Gaulin I thought about this as well, following up Numentas paper for a version implemented in a neural simulator such as NEST and integrated e.g. in the HBP (Human Brain Project) Neurorobotics platform. Would definitely be a very interesting project and maybe make it easier to compare with experimental data/collaborate with neuroscientists.

Kind regards

1 Like