Project : Full-layer V1 using HTM insights


#22

The David Marr Vision book; this has all been done before.
You owe it to yourselves to read it to keep from reinventing the wheel:
http://kryakin.site/David%20Marr-Vision.pdf


#23

Yes, that’s the keeper. If it survives the test of time then that will have been a tremendous help understanding how a 2D map becomes applied to our 3D world. Thanks for adding the paper to this discussion too!

I’m not sure what to make of that one. It’s loaded with information but after scanning for what I in this case need to see accounted for by the word “wave” the only waves mentioned are in regards to sine wave examples. It’s from what I can see missing the kind of spatial reasoning I have been experimenting with and appears to assume a symbolic information processing system.

It may sound like I’m being overly demanding but if the virtual cortical sheet is not lighting up with wave action shown in more recent papers describing results of using new voltage sensitive agents and able to perform as expected in a two frame place avoidance test then it’s not a model that will impress modern neuroscience, therefore the search must go on.


#24

That would be quite typical of me I believe. Jumping to an interpretation from a first photographic impression. Should have pondered about that some more, I’m sorry.
Yet I must say, I was quite impressed by the behavior of that lab rat you modeled, and so I’ve spent quite some time trying to decipher how you did it and what it was all about. In the end I’m not sure I’ve understood much of it. So, I’m still struggling to follow that approach. Please bear with my slow grasp on the matter. Adding to my confusion is that at times you can get quite metaphoric (eg, zombie bots), please consider english is not my mothertongue :wink:

I’ll try to give my understanding another shot, and answer to your post later in the day (And also to your reply on your oscillatory topic). At the moment I’m starting to read Marr’s paper. Thank you again for that gem, @Bitking.


#25

The oscillation has always been in the background of most neuroscience.
It is only very recently that this has been understood to be part of a distinct wave pattern.

While that is important the older work where the oscillator features where a background item are still fully relevant.

I would be wary of forcing wave behavior over focusing on the local functions. I personally think that much of the wave behavior comes from Thalamo-cortico-thalamic connections.


#26

Been off for a while, finished reading David Marr’s Vision, watched several more from the MIT course, and looked at some papers.

That book, Vision, is where Marr develops on his proposition for 3 levels of analysis, which I had to punch somewhat before I was able to integrate it as an interesting and useful viewpoint, or more precisely, an interesting and useful method for expressing (and being aware of) different viewpoints when studying processes.

Carefully studying the visual system as he did, is precisely what I wish to avoid here. But his work on vision, derived from intriguing experiments in psychophysics, is definitely something to read. Most is concerned about how to solve several functional aspects of human vision, and his proposed framework for doing so, with lots related to the definition of a “primal sketch” and study of possible ways of sketching it, all the while solving for, eg., stereoscopy concerns, and amenable to higher level understandings such as adding an egocentric “depth” to the primal sketch (what he calls 2.5D) and inferring whole surfaces before switching to full-blown 3D semantics, which we maybe could call an allocentric representation here.

I’ve also finally dug out some of the work which was trying to explore at V1 dynamics in the same way as I intended. Found those two (related) papers Receptive Field and Feature Map Formation in the Primary Visual Cortex via Hebbian Learning with Inhibitory Feedback and The Dynamics of Image Processing by Feature Maps in the Primary Visual Cortex. Haven’t yet read every detail in those, but there’s already a few things standing out :

  • What’s encouraging, is that the from-scratch formation of orientation selectivity in V1 cells seems quite doable, from simple visual stimuli, and in reach of << quite standard ANN models >>.
  • As a downside, however, the from-scratch formation of orientation selectivity in V1 cells seems quite doable, from simple visual stimuli, and in reach of << quite standard ANN models >>.

So… what to do ? maybe after I learn more about it all, there would be other well-known V1 features which are not captured by these, or shall I look at another approach and try to model concerns more related to SMI or something, like saccades ? or explore more of a hierarchy, I don’t know really.


#27

On the subject of oscillation I recommend the book
Rhythms of the Brain by Gyorgy Buzsaki
Oxford Press 2006


#28

How about a model like this for personal computers?

It would be nice to compare notes with the NEST group. At 2:40 in this video is a researcher modeling the visual cortex of a monkey:

My thoughts are to keep processing time to a minimum by using the usual 2D shock zone environment, which may be tilted to match a 3D terrain. This seems to be closest to how our brain works at the 2D cortical sheet network level. In either case we have many questions.

I would like to invite a guest neuroscientist or two to explain how their model works. We can all go from there. Your thoughts?

Since all “scientific theory” is tentative: whatever as a whole develops in the Numenta forum is still “HTM theory”. The guests would be working on supercomputer sized neuroscientific models that ultimately have to get into the finest of neural detail, which is not the same as HTM theory where there is the added challenge of modeling a whole cortical sheet inside our desktop sized computers. There is no competition that I know of to worry about.


#29

Seems nice !

I’d definitely see the more exchange between such communities the better, but I’m Mr Nobody here. Since their project has been maintained for years, I believe Numenta is already aware it exists.

They claim to support a large number of neuron models, so I guess HTM is amenable to NEST, but NuPIC itself is python based so I don’t know of the benefits it could bring to do such a port (other than obvious positive effect of idea exchange and goodwill synergies). But it seems like it is exactly what @kaikun referred to in the quote you posted, so if there are any volunteers for it already, this could be neat.

As for myself, I’ll try to have a look at some of these links, and the brain-to-robot stuff of the Neurobotics platform is exciting my curiosity, however I know I have great difficulty to function fluently as a lib-user. And all those reference to “Clouds”, “Multi-Device”, “Global”, “Customizable” keywords are for me a no-go : Even if I do realize it offers immense flexibility to a fair number of people, To my mind it indicates that this would turn any issue I could encounter while using the system into an OS-level-configuration issue, for which I have a deep, almost Pavlovian fear.
So, even though I still did not understand your code, Gary, I’m way more confident in my ability to follow your own bitfield-kind-of-reasoning than I’m confident in my ability to use anything like this ^^.

Regards,
Guillaume


#30

Here some notes from what I learned just reading about the problem:

  • The main difference is that NEST is working with spiking neural networks (SNNs), which makes all the simplification of HTM to binary computations much more computational complex.
  • Nevertheless HTM theory comes from neuroscience and the algorithms are designed in a way so they should work for SNNs but adaptation is non-trivial.
  • However in the past the SNNs have lacked scalability and essential features like sufficient plasticity-customization were missing. (This is a general problem in the scientific community and reason why they often switched away from SNNs in e.g. robotics, if they aren’t experimentalists)
  • On the other hand an implementation in NEST can also be compatible with neuromorphic hardware projects like SpiNNaker or BrainScales. This makes it really interesting, even though they add complexity they ultimately are designed to run in parallel, which is hard to archive for HTM on traditional computer/network architectures.

Kind regards


#31

About SNN adaptation : what I got very succinctly from NEST framework presentation is a reference to the fact that they would not typically operate on weight-based models, but rely on much more topology-oriented connectivity lists. Topology is not what the canonical HTM library would do in default “global” mode but is still, in my view, one of the primary strong points of the HTM state of mind : caring first about a topology of dendritic tree and not bother too much about synaptic weight.

As for conversion of a spike “frequency” to a one-bit signal… After seeing one of their visuals which does greatly look like an SDR, could this be for HTM simply a matter of tuning the simulation clock to capture each spike at precise time t as an “on” bit ?

Another idea I had a few days ago to integrate a per-cell scalar information (such as spike frequency) as input to an HTM model was that it could avoid impacting the implementation of the excitatory pathways (ie, not driving higher excitation to postsynaptic pyramidal cells), but rather, to control the level, or extent, of surrounding inhibition.

[Edit]Oh sorry, @kaikun, I think I finally understand what is at stake here. Is it that SNN have progressive increase in depolarization level until they fire at threshold-crossing ? Yes this seems harder to reconcile with HTM indeed.


#32

That’s how I do it. A bit staying on is the same thing as a neuron spiking as fast as it can. If it’s one spike per hundred time cycles there is a 1% duty cycle. This is useful for giving things priority. Whatever most often signals (such as hunger bit or other need) gets most acted upon.


#33

I found a PDF version, and scanned through it a little:

That led me to this one that I now maybe half understand:

We really need a visual of these waves, or something.


#34

I’ve started to read the decades old classic Hubel&Wiesel book, thanks to another link provided by @bitking. I’m almost done with the MIT course, although I’ll probably watch some of them again. In parallel, I’ve scanned through more recent publications.

From all this, a general scheme for the cortical layout of the computer model I intend to write is starting to emerge. Some of it is trying to match with HTM model of excitatory neuron, and some will try additional specifications, mostly related to the handling of topology.

  • I propose that the axonal arbors of excitatory cells are everywhere a fixed and pre-wired setup, that has to be precisely accounted for in the model. They will be allowed to target one or a set of given laminaes in either same cortical sheet as their soma - in which case their source cell position will constrain the extent to which they can transmit information ; or into a distant cortical sheet (eg. axons from LGN to V1, or from V1 to V2) - in which case a topological transform function, however complex, shall be specifiable (V1 to V2 could map a fovea-to-periphery reversion, LGN to V1 should map alternating zebra patterns of ocular dominance, things like that).

  • One very important aspect of this axonal mapping is the amount of overlap, at any particular point of the receiving sheet, from other axons originating from other locations on same source topology. This will have a dramatic impact on both the computation capability of the receiving layer, and the model’s synaptic address footprint for each dendritic segment, see below.

  • In contrast to the fixation of axonal arbors for distinct populations of cells, each excitatory cell dendritic trees seems highly plastic and will be so in the model : It will be allowed any kind of growth or shrinkage, from a (specifiable) default-at-birth which I imagine randomly pre-initialized on the order of 500µm in diameter. In my view, this plasticity is such that I’ve grown the belief that what we distinguish as cell types and layers based on the overall shape of the dendritic arbours is in fact mostly input driven (1).

  • For both biological accuracy and handling the layout size of the model, I propose to push the subdivision of HTM model one step further : what HTM currently defines as segments, I will decompose into subsegments, each with a definite position (2) and taping from laminar-specific inputs (The proximal part operates on a similar layout than distal segments, albeit from a single position fixed by the position of the soma itself). The position of the subsegment will determine which precise input cell it is allowed to retrieve information from, that is, which cells from the source sheet whose axonal arbour overlap the subsegment’s position.

  • This organization above has two effects : First, it is able to more precisely capture the fact, cited in Jeff’s paper, that as few as 8 coincident inputs can trigger a NMDA spike, provided they are close to each other by ~40µm, which is comparable to the extent of my proposed subsegments. Second, by using the (precomputable) reverse-mapping of axon-source-sheet to axon-arbour-center-in-target-sheet, this allows us to consider the task of sampling from source cells whose axonal arbors are ‘overlapping’ at the segment’s position, to simply sampling an area of definite dimension (corresponding to the size of axonal arbors) around that corresponding center in the source sheet.

  • This also enables us to probably fix an upper limit to the address size of each synapse from a given source. With my current synopsis, I believe a total footprint of 16 bits per synapse (up to 12b or 13b address + 4b or 3b stochastic permanency value) is manageable. The remaining offset information to retrieve the actual source candidate are spread out over much coarser subsegment divisions, even coarser per-cell information, or static per-cell-population templates as well as axonal mapping templates.

  • As hinted by the paragraph on dentritic plasticity, a given cell may however decide to tap from several distinct sources - possibly on distinct laminars, but also each laminar (viewed as axonal arbor targets) may hold several axonal mappings (from different sources) together, as long as the sum of distinct synaptic possibilities for a subsegment does not shoot over the 4096 or 8192 candidates for the 12b or 13b address schemes for synapses on each subsegment. According to my first-shot computations, a cell whose dendrites are tapping from a single laminar containing axonal arbor of a single source could still have subsegments able to sense inputs from a lower area spanning around a circle of about 2mm (12b) or 3mm (13b) in diameter over the cortical sheet, even if bijectively mapped to it, which sounds promising for the “integration over wider area” functionality of, say, a V2 relative to a V1.

  • I’m still working on how to convert (or carry on and work with) the fundamentally per-cell-scalar output of LGN, to the proposed binary scheme of HTM. I believe I’ll try some different options at this point.

  • In the case of L4 “simple cells” in V1, In my view, NMDA will probably be a requirement to overshoot the increased threshold from vast correlated inhibition, instead of being a mechanism for prediction as in TM. Such strong input-intensity-correlated inhibition is indeed supposed to play a major role in ensuring that cells respond preferentially to orientation, no matter how faint, and not to intensity itself when less-well oriented. Some “Complex cells” achieving motion direction selectivity, however, could rely on a more TM-like scheme.

Now on a more general note, I’ll start with V1 cells around identified layer 4 and see how they manage to organize orientation selectivity by themselves, (that is, by input-driven progressive reorganization, aka. learning) also in a disposition hopefully resembling the biological one. If I do manage this, I’ll start to add layers one by one, and possibly try to quickly get to L5 with an almost direct mapping to superior colliculus and thus starting to mess with oculomotor stuff.

The simulation of V1 will be that of any primate of the Hominoidea branch (3), either extant, extinct or even imaginary. This is to ensure a V1 layout which will look closely like the human one (4), without necessarily be constrained to be all-alike on the matter of, eg, cortical size. Thus here, V1 could possibly be set as ~3cm across, which for a full scale simulation would bring the number of V1 microcolumns (30µm spaced) to a manageable number of around 1 million on one hemisphere (5).

(1) See the V1 oversizing and multi-lamination of L4 in the most commonly accepted lamination scheme, and eg, the competing Hässler views of what should be called L3 or L4. Also, there is evidence that spiny stellate cells on V1 L4 themselves start out as regular pyramidal cells and progressively take their distinctive form, most probably from the drive of their specific sensory input. I take all those as clues that these laminar concerns are probably developmental and simply dependent on their ambient signal contexts. What seem fixed to specific positions and laminations are the incoming axonal arbors, though, which I’ll try to reflect (and take advantage of) in the model.
(2) position of subsegment is readily identifable as a microcolumn index, and/or offset from microcolumnar indexed position of soma
(3) tail-less monkeys and apes, of which we are a species
(4) although retina and V1 of the quite close-to-us macaque, is already very similar, they seem to have an additional lamination in their V1 “inbox”. This is believed specific to the sister Cercopithecoidea branch to which they belong.
(5) “Manageable” here is to be taken with some reserve : it is likely that for current PC this is still vastly out of reach - and thus I’ll start with a parameter to specify a much smaller extent of the truly simulated area around fovea - but not so much out of reach as to be inimaginable for the near future, or with many computation clusters.


#35

I have been conversing with @gmirey for some time on this general topic. We endeavored to get a sense of the scale of the relative dimensional quantities of the temporal layer, inputs and cell/column spacing, number and range of the connections.

You know - the basic facts of making a realistic model. Some of what was discussed was what is known about reach and density of the dendritic arbor in the “input” layer.

We established the key factors would be: what is the spacing of the ascending corticocortical axonal bundles, what is the spacing of the columns/cell bodies, what is the dendritic density, what is the synapse density.

I did a little digging and came up with this:
We are trying to describe a complex 3-dimensional structure composed of messy biological bits.
You indicated that you will be back for layer 2/3 at some future time so let’s focus on lower layers.


Looking at the input corticocortical connections - how apart are they? As indicated earlier - we will skip the massive thalamocortical arbors as I don’t think that you will have to model that as an information path.
These massive thalamocortical arbors are a shotgun to insure that layer IV is induced into resonance with the thalamus

We have to account for cell body spacing, ascending axon bundle spacing, dendrite reach & dendrite arbor spatial density. Fortunately, I am finding papers that offer some figures on all of these items.

Note the massive inter-layer local projections from deep pyramidal axons. Keep in mind that they primarily project on inhibitory inter-neurons, suppressing the losers in local recognition contests.

Numerous papers seem to agree that the spread of the layer 2/3 and lower layers dendrites (radius) is about 300 micrometers (0.3 mm) that gives a diameter of about 0.6 mm at the extream tips of the dendrites. This is the long tails (in the truest sense of the word) and the average length is somewhat shorter. Also - the dendrites don’t shoot out in a straight path so the 0.5 mm shown in this diagram is a better maximum figure.
See this for more details:


The spatial density of the branching dendrites tend s to fill space in a constant density.

so how many synapses are there for a unit of space?


It varies but let’s say 1 per cubic micrometer.

While we are at it - what is the density of the microcolumns and cell bodies?

Tissue Property or Measurement Value
Y, average interneuron distance. 20.0 μm (1) (estimated)
P, average intercolumn distance. 26.1 μm (1)
ρ, slide neuronal density 0.0013 neurons/μm2 (1)
l 341 μm (1)
s, thickness of the thin slice 30 μm (1)
radius or neurons (average) 5 μm
% interneurons 20% (2)
Model Parameter Value
dn, interneuron distance. 23.1 μm
dc, intercolumn distance. 29 μm
θ Uniform random [0, 2π]
φ Uniform random, [0, π/3]
% omitted neurons 40%
δdn Gaussian distribution, σ=4.7μm
δxn, δzn Uniform random, [−6 μm, 6 μm]
δxc, δzc Uniform random, [−6 μm, 6μm]
N, number of images for average. 500

How handy is that?
The microcolumns are on ~ 26-30 micrometer spacing.

Now we have to match that up with the ascending axonal bundle spacing that pokes through the dendrite arbors. You are looking at area 17 and the Hubel & Weisel paper has a lot to say about what goes on there.
so does this paper:
Organization of pyramidal neurons in area 17 of monkey visual cortex Alan Peters Claire Sethares
Sadly - behind a paywall.
The abstract is very helpful:
Abstract

In sections of area 17 of monkey visual cortex treated with an antibody to MAP2 the disposition of the cell bodies and dendrites of the neurons is readily visible. In such preparations, it is evident that the apical dendrites of the pyramidal cells of layer VI form fascicles that pass into layer IV, where most of them gradually taper and form their terminal tufts. In contrast, the apical dendrites of the smaller layer V pyramidal cells come together in a more regular fashion. They form clusters that pass through layer IV and into layer II/III where the apical dendrites of many of the pyramidal cells in that layer add to the clusters. In horizontal sections taken through the middle of layer IV, these clusters of apical dendrites are found to have an average center‐to‐center spacing of about 30 μm, and it is proposed that each cluster of apical dendrites represents the axis of a module of pyramidal cells that has a diameter of about 30 μm and contains about 142 neurons.

Is this typical for all areas of the cortex? Unfortunately, this useful paper is also behind a paywall.
The Organization of Pyramidal Cells in Area 18 of the Rhesus Monkey - Alan Peters, J. Manuel Cifuentes and Claire Sethares
Maybe you can hunt this down on your own. It has this handy histogram:
Nearest%20axonal%20neighbor%20histogram

So - 20 to 30 micrometer spacing on the axonal bundles too. I suppose that makes sense that it matches up with the microcolumns. That 30-micrometer spacing seems to turn up a lot in these papers.

Like you I have to see pictures to fix this in my mind. 30 um out of maybe 500 um is a little less than 10 percent of the dendrite arbor field. I think it looks something like this:


Yes - before you jump on me - the dendrite arbors should be denser but the picture is too complicated to make out then.

@gmirey responded with this excellent paper:
https://www.ncbi.nlm.nih.gov/pubmed/15260960
and this one:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4370903/

Working with what we have so far:

Since we are visually oriented - a Graphic solution:
30 µm spacing of cell bodies
500 µm / 30 µm = 16.6
Rounding up = 17 = diameter of dendrite reach expressed in terms of cell bodies.
Set up a repeating field of cell bodies … inscribe a 500 µm circle
inspect and remove the cells x 4 corners in the local area…
Any blue cell reaches the center of the inscribed circle.
Since this is a repeating pattern any starting point has the same solution.
The overlap of dendritic reach is:
(17 x 17) – (4 x 13)
289 - 52 = 237 overlapping cells at any given point.
An inscribed area of 500 µm diameter circle gives an area of πr²
3.141 * (250 * 250) = 196350 µm² = input area for any cell body
Assuming that the layer is I µm thick for a first approximation (prolly much bigger!)
Going with the 1 synapse per µm³ and dividing by the 237 overlapping fields
196350 µm² / 237 = 828 synapses per cell per 1 µm of thickness.

It is likely that the dendrites are spaced out through the thickness of the layer. It is very likely that this density is shared with all the layers. (2/3, 4, 6)

dendrite%20area

So just how thick is layer 1, also knows as the Molecular layer (lamina molecular)
Here is a microphotograph to help us out - with a very helpful size bar:


http://www.anatomyatlases.org/MicroscopicAnatomy/Section06/Plate0697.shtml
On my screen the 100 µm bar is .795" and the layer thickness measures 3.75" so doing the math gives a layer 1 thickness of 470 um

Plugging that into whole 828 synapses per cell per 1 µm of thickness if one layer was doing it all works out to 828 synapses x 470 um = 389,160 synapse per dendrite field.

Let’s assume that each of the main layers represented here (2/3, 4, 6) gets about a third of the dendrite space so:
389,160 synapse / 3 = 129720 synapses spines per cell per layer type. This seems about 10 times larger than I have read elsewhere so I must be screwing up somewhere.


At how many different minicolumns is connected an unique neuron in a minicolumn?
#36

Yes, this exchange was very fruitful and @bitking insights are invaluable.

My model values differ slighly from those of last blue square, since I’m still attached to the hex lattice to layout even the microcolumnar structure, both from personal preference and from the belief that it will help quick-compute indexes on roughly circular areas to account for both those dendritic and axonal arbors concerns.

I’ll try to come with details on some of my latest figures with the hex lattice, and specifics of most of the layout, hopefully with a few drawings as well… sorry for the dense walls of text to this point. So thank you @bitking for those figures, discussion, infos, and now for bringing color to this topic :wink:


#37

Hi everyone. This time I also tried some visualizations.

First, meet the hex lattice

HexLattice
This here could very well be a top-down view of any cortical sheet model. Each microcolumn center would be identified with one of these points.

Each dot here has an equal spacing with any of its neighbors, and there are no ‘cornering’ neighbours distinct from ‘side’ neighbours, as would be the case with a square tiling scheme. I think it makes sense for an area so heavily concerned with topology as is V1. And in fact for any cortical area if we take the view that they all operate on same algorithm.

Now, we could object that using this weird layout seems to be lots of pain for little gain, however as shown below, there are very straightforward schemes to bind our common indexing for two-dimensional arrays to this:

HexLatticeIndexing
You see that the grid above is still a common, regular, 2D addressable thing. Only skewed a little when representing it on display, or taking topology itself into account, so this layout has almost no impact on iteration times or global indexing complexity.

Now, an additional benefit is that any regular neighborhood is already a pretty good approximation of a circular area. Below is an illustration of an area which spans 4 neighbors from its center point, so this is a “9-across” area:

HexLatticeNeighbourhood
I have a quick and dirty Excel sheet at my side, which tells me that there are 61 distinct points inside this shape. And it will give me that number for any of the “neighbours from center” values. You can count them here if you wish.
With a few tweaks, you can fast-compute any offset-from the center (in the global addressing scheme described as a regular (x;y) index above) from any of these 0…60 possible address values. And this holds true at any scale, thus increasing the speed at which we can compute the extent, retrieve a point address, or iterate upon any of these roughly circular areas, of any diameter.

So, an index taken as an “offset-from-center” in the figure above would fit in only 6 bits. But obviously, from the reflection and estimations we carried around together with @bitking, we’d need to describe much larger areas for most of our concerns (which are both sides of the following matter : the extent of the axonal tree on one hand, and the dendritic extent on the other hand).

So let’s try to describe the “matter” in more detail.
You all know about the main excitatory cell of the cortical column : The pyramidal cell, and its HTM interpretation:

PCandHTM

Let’s add some rulers around this one guy, or one of its close cousins…

PCwithLegends

Now that ~30µm for microcolumn-to-microcolumn center we settled on with @bitking is maybe not a figure everybody would agree on. It is on the lower end of the proposed scales from wikipedia (up to ~50 or even ~80µm). I believe however that, if one would take a greater figure than ~30µm, the philosophy of the proposed organization would not be much altered, and it would only reduce the count of stuff-per-area that we need to consider. So, ~30µm seems like the “worst case” scenario.

Each soma of a pyramidal cell is set on one microcolumnar index, on the hex layout. Typical horizontal dendritic extent (whether basal or apical) is at its ease in a 8-neighbourhood (17-across), but dendritic plasticity would allow them to grow up to a 12-neighborhood (25-across) if really information-starved. My Excell sheet tells me that there are 217 distinct possibilities for the ‘typical’ case, and up to 469 distinct possibilities when stretched. Those require at worse 9 bits to localize a subsegment relative to its cell’s soma position.

DendriticExtent

We can thus fully encode a 3D position for the subsegment on 16 bits, if we take 7 more bits for the vertical, giving us 128 possible “input template” per cortical area. Now what’s an input template ? It is a description of the axonal side of the problem. These subsegments, as exemplified by the little part magnified in the blue box two pics above, are highly localized. Beside allowing the finer 8-coincidency threshold detection I propose, this high specificity of localization is the whole point of them.

An “input template” is localized in the vertical axis of the receiving sheet, and may allow several “sources”. A source is : a “sheet” of origin (same or other cortical area, or even a deeper part of the brain such as LGN), and a specific “population” of cells within that sheet, sending axons towards the receiving area. Now, for each given source, there is in the model a precise topological relationship to the receiving area, which allow us to :

  • define topologies as finely as the biological wiring we wish to simulate would require
  • put an upper limit to the (otherwise huge) address of input cell, as viewed from a subsegment.

We’ll take a slice below to keep things simple in an example of topological mapping :

TopoMapping

Maybe this looks like it is unecessarily complicated at this point, however now the wiring is completely specified. For a dendrite subsegment sampling the received input, the extent is applied the inverse scaling of the original mapping. In that example above, this would be divided by two, bringing the sampled neighbourood (around associated center) to two, ie. a 5-across area. You can check this on the above image, imagining how far apart on the source sheet could two cells having overlapping axonal arbors on the receiving sheet be situated. Any position in the original sheet layout thus gets a restricted area of incoming axonal overlap. For the (very lightweight) example above, it would look like this:

AxonalSampling

Now, the situation gets a little more complicated when the mapping is non-trivial, such as for ocular dominance stripes we have in V1, however I believe this is manageable.

Now, If we allow 12b or 13b per-synapse address, given a single sub-segment as proposed in my previous post, maybe there will be cases where the address-space is getting somewhat tight to allow multi-source with large axonal overlaps. Since I’m not confident we know of all weird-wiring cases which could potentially come up in the brain, let’s say we increase that value to a very conservative 16b per synapse… I’ll take whatever estimate you guys deem accurate for the max number of synapses in a subsegment having a length in the order of 40µm, but I believe this could be quite reasonable at this point.

At any rate, given a full-fledged ~6000 synapses cell, even with low filling-efficiency of subsegments at around 75% (this would bring the total synaptic “slot” count to ~8000), with 16b per synapse address and 4b for permanency value (permanency stored in another contiguous array, still very low profile as they’ll be stochastically updated), this makes the footprint of a cell at around 20KB.
The full connectivity state of a 100 thousand-cell simulation can thus be stored in around 2GB.

Seems high still ? Maybe. However the beauty of the topological structure is, no matter how many areas or minicolumns-per-area you’d wish to reasonably simulate, those 16b-address per synapse-on-given-subsegment values would never change. Were it computationally anywhere in reach, a full scale brain simulation could thus, in my view, still operate with those same values.

Which seems reasonable enough to start opening Visual Studio and work from there for the V1 sim. Now I have some work on my hands :wink:


#38

Excellent breakdown and visualizations. One thing that pops out to me is a potential memory optimization from applying a strategy like this to the Spatial Pooling algorithm. Each input bit could have random potential synapses that are addressed in “axonal arbour” coordinate space. Then given a target position in a destination sheet can become a reference point used to iterate the minicolumns which are referenced by those potential synapses. Think I’ll see if I can work this into HTM.go.


#39

Sure, HTM at the moment does not lean heavily towards topology concerns, that’s why we could maybe see this stuff mostly as an “extensional” specification, which I believe could be integrated within current HTM Pooling or even TM algorithms, maybe with some addos for ‘t-1’ concerns (I think I will have to address t-1 also in the quite near future, maybe as what I’d call a distinct ‘population’ in the input template).
So, if you can get something running quickly along these lines and smoothly integrated with HTM that would be neat :slight_smile:


#40

Yes, I’m also thinking on how to apply it to TM algorithm. One complexity (besides T-1) is the distinction of multiple segments per cell. From a certain perspective, that could just be an abstraction born from a lack of topology in the algorithm (as you pointed out, Jeff always describes the distal synapses as coincidence detectors, activated when several of them that are close to other in space and time are active). It may not be necessary to model distinct segments in an implementation of TM that includes topology.


#41

Perhaps topology is not necessary for TM.
For my part my first “bet” in the simulation would be to consider that, besides position, dynamic input context, and output axonal wiring, nothing really distinguishes a pyramidal cell from another. So what must be true of a V1 L4 stellate, would be true of the cells which achieve TM. And I’ll try to fit-all in that scheme above.
I’m quite impatient of hearing back from your results, though :slight_smile: