Possiblity of global inhibition in Temporal Memory

Science Numenta Theory
1

So. One of the problem that I met when I’m messing with NLP in HTM is that Temporal Memory although predicts all possible future. It doesn’t give me any sense of probability.

For example, this is the SDR that TM is predicting after feeding it a few letters. (The number indicates how many bit are on in a given category)

{ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
16., 0., 0., 0., 0., 0., 16., 0., 0., 0., 0., 0., 0., 0., 0.}

As you see. In the predicted SDR. Either all bits are off or all 16 bits are on. This property of TM removes the ability for me to estimate how strongly the next character possibly is. It would be nice that TM tells me that category 15 is possible but I’m not that sure. Instead of “Category 15 is possible above the threshold. So I guess It will be present.”

And so I purpose an solution. I think that mechanisms like global inhibition in the SP can help with this problem. By limiting to only allow the top neurons activating. TM can express probability by turning off some neurons that it has lower confident with.

Well… Is global inhibition in TM biologically even biologically possible? What else could such mechanisms imply?

1 Like
2

I don’t have the expertise to answer your specific question, but I can propose an alternate solution to the problem, which is voting. In a complex system like a brain, a particular cortical column is never operating in isolation. Signals are being integrated from up, down, and sideways through the hierarchy, and everyone is voting together on what is beign experienced. I think this is what gives one prediction preference over another. In the case where there is still ambiguity, I also propose keeping track of how recently a neuron has fired and selecting the one which fired least recently (to simulate cell fatigue)

3

Thanks.

I have tested simulating cell fatigue (with 1 TM). It doesn’t work. TM gives up predicting anything after a few characters. I’ll try to make a voting version.

4

I have read some of this paper, entitled Orientation Selectivity and the Arrangement of Horizontal Connections in Tree Shrew Striate Cortex (thanks @bitking), and it certainly shows evidence of horizontal connections seemingly stronger in some areas and perhaps inhibited in others?

IMG_E432067F5307-1
IMG_E432067F5307-11837×1564 2.78 MB

5

@rhyolight I see that those lateral connections are excitatory for longer range connections.
The inhibitory inter-neurons are active at shorter ranges.

The combination can form Garbor filters or hex-grid cell structures depending on the relative distances.

This paper gives excellent details of the long-range connections:
https://www.researchgate.net/publication/12675879_Horizontal_Synaptic_Connections_in_Monkey_Prefrontal_Cortex_An_In_Vitro_Electrophysiological_Study

6

Thank you for that correction. I sometimes get distracted by colorful pictures.

1 Like