What is the best way to represent a set of synapses per dendrite per cell?

Engineering
1

What data structures or types or objects are the best to implement synpases on a dendrite segment on a cell and thus potentially create an entire cortical columnar model? The algorithms can be boiled down without actually creating every biological unit but what if we were to make a programming model that replicates a cortical column entirely? How can we represent each unit while managing memory and performance?

Are there any specific abstractions one should implement over these units to make them more program feasible?

P.S.: New to the theory and trying to implement it ground up.

1 Like
2

Perhaps this?

1 Like
3

I think we use arrays of arrays. First, a cell can have an array of segments. Each segment can have one or more synapses, which are permanence values.

4

What is different with the approach I am proposing is that for a given cell body synapses with a strength of zero do not have to be maintained or processed in all operations. A pre-scan operation could extract the active connections. This exploits an essential feature of sparsity - only active connections are considered. The path table holds all the potential connections but only the active connections are maintained with each cell body data structure. This could result in a dramatic speedup in certain parts of processing.

There are also certain desirable biological features that are simplified as described in the original post.

If you choose to keep the strength of every synapse, active or not, you do not have to have to keep the location as this is stored in the related path table resulting in a reduction of the data consumed by the pointer to the synapse.

It would also be possible to make a path table that is a circular distribution around the cell body to facilitate a signal sampling that looks a little more biological. Topology counts.

1 Like
5

Thank you @Bitking and @rhyolight
It seems interesting. Will look into SOMs and storage mechanisms. I initially thought of storing all synapses externally and then collecting the data for every operation. Since memory optimisation is a concern, using arrays, like @rhyolight mentioned was given a priority. @nivedita and I are using Julia for the implementation and we think the performance will be considerably good after looking at certain benchmarks and testing some code.

I think storing pathways like @Bitking mentioned can be a good addition to the mechanism described in this post:
https://discourse.numenta.org/t/a-different-point-of-view-on-building-ai-system/3097

Is there an even better way to encode these biological units and functions in a workable low-memory abstraction?

6

You might look up Cells4.cpp in nupic.core. I think we have specific code in there to do some matrix math. Is that right @mrcslws?

1 Like
7

Thank you for pointing that out. I’ll look into it for reference.

8

From a computational point of view, it is important to be able to quickly access synapses from both the pre-synaptic and post-synaptic side. This can amount to some extra book keeping but is well worth the programming effort and bytes of RAM.

Two required capabilities of HTM synapses are:

  1. Every active input needs to be able to quickly notify every connected post-synaptic site.
  2. Every learning output site needs to be able to quickly see which inputs are connected to it.

Scanning through all of the synapses looking for things takes a long time compared to already having the connectivity data in two arrays-of-arrays which are indexed by the pre-synapse and post-synapse.

2 Likes