Non-binary connections

I’m working on my own version of a temporal pooler, and I wonder… how sacrilegious is the idea of a weighted synapse?

My idea for TP is to simultaneously train on the TP’s input as well as its own prior activity, with strongly weighted self-reinforcing connections. Does that vibe with the core principles of HTM theory?

Edit:spelling

Idk if it explicitly violates them, but I wouldn’t let that deter you from trying anything.
The main things I would be sure and stick too are the concepts of localization and modularity.
So there’s no master or global anything.

Each region can only represent a certain slice of sensory space, and each column and cell has a limited receptive field.

A TP region should (I think) function very much like a usual sensory region. But its role and behavior can change by changing how it’s receptive field works. For instance:

  • monitoring sensory region(s) along with/instead of the raw sensory data

  • monitoring its input over longer time periods (like including active/winner/pred cells from before t-1)

1 Like

Individual synapses can be considered as binary if you are only concerned with their connected status. However, in modeling the network we might also want to consider the amount of influence the firing of one neuron has on another. Numerically, this can be represented by a scalar, but also as a sum of unitary input bits. The latter interpretation would correspond to multiple synapse connections between the axon and dendrite arbors between two neurons. So, I think there is some plausible biological justification for using non binary weights to describe influence between neurons.

1 Like

Cool!

I’ve built a few prototypes of TP’s.
Your idea sounds similar to what I ended up with.
I made a (20 minute) presentation where I explain one way to make a TP:

Although the presentation starts by talking about “grid cells”, I discuss the TP for the last few minutes.
I hope this helps.

2 Likes