The David Marr Vision book; this has all been done before.
You owe it to yourselves to read it to keep from reinventing the wheel:
http://kryakin.site/David%20Marr-Vision.pdf
2 Likes
Yes, thatās the keeper. If it survives the test of time then that will have been a tremendous help understanding how a 2D map becomes applied to our 3D world. Thanks for adding the paper to this discussion too!
Iām not sure what to make of that one. Itās loaded with information but after scanning for what I in this case need to see accounted for by the word āwaveā the only waves mentioned are in regards to sine wave examples. Itās from what I can see missing the kind of spatial reasoning I have been experimenting with and appears to assume a symbolic information processing system.
It may sound like Iām being overly demanding but if the virtual cortical sheet is not lighting up with wave action shown in more recent papers describing results of using new voltage sensitive agents and able to perform as expected in a two frame place avoidance test then itās not a model that will impress modern neuroscience, therefore the search must go on.
That would be quite typical of me I believe. Jumping to an interpretation from a first photographic impression. Should have pondered about that some more, Iām sorry.
Yet I must say, I was quite impressed by the behavior of that lab rat you modeled, and so Iāve spent quite some time trying to decipher how you did it and what it was all about. In the end Iām not sure Iāve understood much of it. So, Iām still struggling to follow that approach. Please bear with my slow grasp on the matter. Adding to my confusion is that at times you can get quite metaphoric (eg, zombie bots), please consider english is not my mothertongue 
Iāll try to give my understanding another shot, and answer to your post later in the day (And also to your reply on your oscillatory topic). At the moment Iām starting to read Marrās paper. Thank you again for that gem, @Bitking.
The oscillation has always been in the background of most neuroscience.
It is only very recently that this has been understood to be part of a distinct wave pattern.
While that is important the older work where the oscillator features where a background item are still fully relevant.
I would be wary of forcing wave behavior over focusing on the local functions. I personally think that much of the wave behavior comes from Thalamo-cortico-thalamic connections.
1 Like
Been off for a while, finished reading David Marrās Vision, watched several more from the MIT course, and looked at some papers.
That book, Vision, is where Marr develops on his proposition for 3 levels of analysis, which I had to punch somewhat before I was able to integrate it as an interesting and useful viewpoint, or more precisely, an interesting and useful method for expressing (and being aware of) different viewpoints when studying processes.
Carefully studying the visual system as he did, is precisely what I wish to avoid here. But his work on vision, derived from intriguing experiments in psychophysics, is definitely something to read. Most is concerned about how to solve several functional aspects of human vision, and his proposed framework for doing so, with lots related to the definition of a āprimal sketchā and study of possible ways of sketching it, all the while solving for, eg., stereoscopy concerns, and amenable to higher level understandings such as adding an egocentric ādepthā to the primal sketch (what he calls 2.5D) and inferring whole surfaces before switching to full-blown 3D semantics, which we maybe could call an allocentric representation here.
Iāve also finally dug out some of the work which was trying to explore at V1 dynamics in the same way as I intended. Found those two (related) papers Receptive Field and Feature Map Formation in the Primary Visual Cortex via Hebbian Learning with Inhibitory Feedback and The Dynamics of Image Processing by Feature Maps in the Primary Visual Cortex. Havenāt yet read every detail in those, but thereās already a few things standing out :
- Whatās encouraging, is that the from-scratch formation of orientation selectivity in V1 cells seems quite doable, from simple visual stimuli, and in reach of << quite standard ANN models >>.
- As a downside, however, the from-scratch formation of orientation selectivity in V1 cells seems quite doable, from simple visual stimuli, and in reach of << quite standard ANN models >>.
Soā¦ what to do ? maybe after I learn more about it all, there would be other well-known V1 features which are not captured by these, or shall I look at another approach and try to model concerns more related to SMI or something, like saccades ? or explore more of a hierarchy, I donāt know really.
1 Like
On the subject of oscillation I recommend the book
Rhythms of the Brain by Gyorgy Buzsaki
Oxford Press 2006
How about a model like this for personal computers?
It would be nice to compare notes with the NEST group. At 2:40 in this video is a researcher modeling the visual cortex of a monkey:
My thoughts are to keep processing time to a minimum by using the usual 2D shock zone environment, which may be tilted to match a 3D terrain. This seems to be closest to how our brain works at the 2D cortical sheet network level. In either case we have many questions.
I would like to invite a guest neuroscientist or two to explain how their model works. We can all go from there. Your thoughts?
Since all āscientific theoryā is tentative: whatever as a whole develops in the Numenta forum is still āHTM theoryā. The guests would be working on supercomputer sized neuroscientific models that ultimately have to get into the finest of neural detail, which is not the same as HTM theory where there is the added challenge of modeling a whole cortical sheet inside our desktop sized computers. There is no competition that I know of to worry about.
Seems nice !
Iād definitely see the more exchange between such communities the better, but Iām Mr Nobody here. Since their project has been maintained for years, I believe Numenta is already aware it exists.
They claim to support a large number of neuron models, so I guess HTM is amenable to NEST, but NuPIC itself is python based so I donāt know of the benefits it could bring to do such a port (other than obvious positive effect of idea exchange and goodwill synergies). But it seems like it is exactly what @kaikun referred to in the quote you posted, so if there are any volunteers for it already, this could be neat.
As for myself, Iāll try to have a look at some of these links, and the brain-to-robot stuff of the Neurobotics platform is exciting my curiosity, however I know I have great difficulty to function fluently as a lib-user. And all those reference to āCloudsā, āMulti-Deviceā, āGlobalā, āCustomizableā keywords are for me a no-go : Even if I do realize it offers immense flexibility to a fair number of people, To my mind it indicates that this would turn any issue I could encounter while using the system into an OS-level-configuration issue, for which I have a deep, almost Pavlovian fear.
So, even though I still did not understand your code, Gary, Iām way more confident in my ability to follow your own bitfield-kind-of-reasoning than Iām confident in my ability to use anything like this ^^.
Regards,
Guillaume
Here some notes from what I learned just reading about the problem:
- The main difference is that NEST is working with spiking neural networks (SNNs), which makes all the simplification of HTM to binary computations much more computational complex.
- Nevertheless HTM theory comes from neuroscience and the algorithms are designed in a way so they should work for SNNs but adaptation is non-trivial.
- However in the past the SNNs have lacked scalability and essential features like sufficient plasticity-customization were missing. (This is a general problem in the scientific community and reason why they often switched away from SNNs in e.g. robotics, if they arenāt experimentalists)
- On the other hand an implementation in NEST can also be compatible with neuromorphic hardware projects like SpiNNaker or BrainScales. This makes it really interesting, even though they add complexity they ultimately are designed to run in parallel, which is hard to archive for HTM on traditional computer/network architectures.
Kind regards
About SNN adaptation : what I got very succinctly from NEST framework presentation is a reference to the fact that they would not typically operate on weight-based models, but rely on much more topology-oriented connectivity lists. Topology is not what the canonical HTM library would do in default āglobalā mode but is still, in my view, one of the primary strong points of the HTM state of mind : caring first about a topology of dendritic tree and not bother too much about synaptic weight.
As for conversion of a spike āfrequencyā to a one-bit signalā¦ After seeing one of their visuals which does greatly look like an SDR, could this be for HTM simply a matter of tuning the simulation clock to capture each spike at precise time t as an āonā bit ?
Another idea I had a few days ago to integrate a per-cell scalar information (such as spike frequency) as input to an HTM model was that it could avoid impacting the implementation of the excitatory pathways (ie, not driving higher excitation to postsynaptic pyramidal cells), but rather, to control the level, or extent, of surrounding inhibition.
[Edit]Oh sorry, @kaikun, I think I finally understand what is at stake here. Is it that SNN have progressive increase in depolarization level until they fire at threshold-crossing ? Yes this seems harder to reconcile with HTM indeed.
Thatās how I do it. A bit staying on is the same thing as a neuron spiking as fast as it can. If itās one spike per hundred time cycles there is a 1% duty cycle. This is useful for giving things priority. Whatever most often signals (such as hunger bit or other need) gets most acted upon.
I found a PDF version, and scanned through it a little:
That led me to this one that I now maybe half understand:
We really need a visual of these waves, or something.
1 Like
Iāve started to read the decades old classic Hubel&Wiesel book, thanks to another link provided by @bitking. Iām almost done with the MIT course, although Iāll probably watch some of them again. In parallel, Iāve scanned through more recent publications.
From all this, a general scheme for the cortical layout of the computer model I intend to write is starting to emerge. Some of it is trying to match with HTM model of excitatory neuron, and some will try additional specifications, mostly related to the handling of topology.
-
I propose that the axonal arbors of excitatory cells are everywhere a fixed and pre-wired setup, that has to be precisely accounted for in the model. They will be allowed to target one or a set of given laminaes in either same cortical sheet as their soma - in which case their source cell position will constrain the extent to which they can transmit information ; or into a distant cortical sheet (eg. axons from LGN to V1, or from V1 to V2) - in which case a topological transform function, however complex, shall be specifiable (V1 to V2 could map a fovea-to-periphery reversion, LGN to V1 should map alternating zebra patterns of ocular dominance, things like that).
-
One very important aspect of this axonal mapping is the amount of overlap, at any particular point of the receiving sheet, from other axons originating from other locations on same source topology. This will have a dramatic impact on both the computation capability of the receiving layer, and the modelās synaptic address footprint for each dendritic segment, see below.
-
In contrast to the fixation of axonal arbors for distinct populations of cells, each excitatory cell dendritic trees seems highly plastic and will be so in the model : It will be allowed any kind of growth or shrinkage, from a (specifiable) default-at-birth which I imagine randomly pre-initialized on the order of 500Āµm in diameter. In my view, this plasticity is such that Iāve grown the belief that what we distinguish as cell types and layers based on the overall shape of the dendritic arbours is in fact mostly input driven (1).
-
For both biological accuracy and handling the layout size of the model, I propose to push the subdivision of HTM model one step further : what HTM currently defines as segments, I will decompose into subsegments, each with a definite position (2) and taping from laminar-specific inputs (The proximal part operates on a similar layout than distal segments, albeit from a single position fixed by the position of the soma itself). The position of the subsegment will determine which precise input cell it is allowed to retrieve information from, that is, which cells from the source sheet whose axonal arbour overlap the subsegmentās position.
-
This organization above has two effects : First, it is able to more precisely capture the fact, cited in Jeffās paper, that as few as 8 coincident inputs can trigger a NMDA spike, provided they are close to each other by ~40Āµm, which is comparable to the extent of my proposed subsegments. Second, by using the (precomputable) reverse-mapping of axon-source-sheet to axon-arbour-center-in-target-sheet, this allows us to consider the task of sampling from source cells whose axonal arbors are āoverlappingā at the segmentās position, to simply sampling an area of definite dimension (corresponding to the size of axonal arbors) around that corresponding center in the source sheet.
-
This also enables us to probably fix an upper limit to the address size of each synapse from a given source. With my current synopsis, I believe a total footprint of 16 bits per synapse (up to 12b or 13b address + 4b or 3b stochastic permanency value) is manageable. The remaining offset information to retrieve the actual source candidate are spread out over much coarser subsegment divisions, even coarser per-cell information, or static per-cell-population templates as well as axonal mapping templates.
-
As hinted by the paragraph on dentritic plasticity, a given cell may however decide to tap from several distinct sources - possibly on distinct laminars, but also each laminar (viewed as axonal arbor targets) may hold several axonal mappings (from different sources) together, as long as the sum of distinct synaptic possibilities for a subsegment does not shoot over the 4096 or 8192 candidates for the 12b or 13b address schemes for synapses on each subsegment. According to my first-shot computations, a cell whose dendrites are tapping from a single laminar containing axonal arbor of a single source could still have subsegments able to sense inputs from a lower area spanning around a circle of about 2mm (12b) or 3mm (13b) in diameter over the cortical sheet, even if bijectively mapped to it, which sounds promising for the āintegration over wider areaā functionality of, say, a V2 relative to a V1.
-
Iām still working on how to convert (or carry on and work with) the fundamentally per-cell-scalar output of LGN, to the proposed binary scheme of HTM. I believe Iāll try some different options at this point.
-
In the case of L4 āsimple cellsā in V1, In my view, NMDA will probably be a requirement to overshoot the increased threshold from vast correlated inhibition, instead of being a mechanism for prediction as in TM. Such strong input-intensity-correlated inhibition is indeed supposed to play a major role in ensuring that cells respond preferentially to orientation, no matter how faint, and not to intensity itself when less-well oriented. Some āComplex cellsā achieving motion direction selectivity, however, could rely on a more TM-like scheme.
Now on a more general note, Iāll start with V1 cells around identified layer 4 and see how they manage to organize orientation selectivity by themselves, (that is, by input-driven progressive reorganization, aka. learning) also in a disposition hopefully resembling the biological one. If I do manage this, Iāll start to add layers one by one, and possibly try to quickly get to L5 with an almost direct mapping to superior colliculus and thus starting to mess with oculomotor stuff.
The simulation of V1 will be that of any primate of the Hominoidea branch (3), either extant, extinct or even imaginary. This is to ensure a V1 layout which will look closely like the human one (4), without necessarily be constrained to be all-alike on the matter of, eg, cortical size. Thus here, V1 could possibly be set as ~3cm across, which for a full scale simulation would bring the number of V1 microcolumns (30Āµm spaced) to a manageable number of around 1 million on one hemisphere (5).
(1) See the V1 oversizing and multi-lamination of L4 in the most commonly accepted lamination scheme, and eg, the competing HƤssler views of what should be called L3 or L4. Also, there is evidence that spiny stellate cells on V1 L4 themselves start out as regular pyramidal cells and progressively take their distinctive form, most probably from the drive of their specific sensory input. I take all those as clues that these laminar concerns are probably developmental and simply dependent on their ambient signal contexts. What seem fixed to specific positions and laminations are the incoming axonal arbors, though, which Iāll try to reflect (and take advantage of) in the model.
(2) position of subsegment is readily identifable as a microcolumn index, and/or offset from microcolumnar indexed position of soma
(3) tail-less monkeys and apes, of which we are a species
(4) although retina and V1 of the quite close-to-us macaque, is already very similar, they seem to have an additional lamination in their V1 āinboxā. This is believed specific to the sister Cercopithecoidea branch to which they belong.
(5) āManageableā here is to be taken with some reserve : it is likely that for current PC this is still vastly out of reach - and thus Iāll start with a parameter to specify a much smaller extent of the truly simulated area around fovea - but not so much out of reach as to be inimaginable for the near future, or with many computation clusters.
1 Like
I have been conversing with @gmirey for some time on this general topic. We endeavored to get a sense of the scale of the relative dimensional quantities of the temporal layer, inputs and cell/column spacing, number and range of the connections.
You know - the basic facts of making a realistic model. Some of what was discussed was what is known about reach and density of the dendritic arbor in the āinputā layer.
We established the key factors would be: what is the spacing of the ascending corticocortical axonal bundles, what is the spacing of the columns/cell bodies, what is the dendritic density, what is the synapse density.
I did a little digging and came up with this:
We are trying to describe a complex 3-dimensional structure composed of messy biological bits.
You indicated that you will be back for layer 2/3 at some future time so letās focus on lower layers.
Looking at the input corticocortical connections - how apart are they? As indicated earlier - we will skip the massive thalamocortical arbors as I donāt think that you will have to model that as an information path.
These massive thalamocortical arbors are a shotgun to insure that layer IV is induced into resonance with the thalamus
We have to account for cell body spacing, ascending axon bundle spacing, dendrite reach & dendrite arbor spatial density. Fortunately, I am finding papers that offer some figures on all of these items.
Note the massive inter-layer local projections from deep pyramidal axons. Keep in mind that they primarily project on inhibitory inter-neurons, suppressing the losers in local recognition contests.
Numerous papers seem to agree that the spread of the layer 2/3 and lower layers dendrites (radius) is about 300 micrometers (0.3 mm) that gives a diameter of about 0.6 mm at the extreme tips of the dendrites. This is the long tails (in the truest sense of the word) and the average length is somewhat shorter. Also - the dendrites donāt shoot out in a straight path so the 0.5 mm shown in this diagram is a better maximum figure.
See this for more details:
The spatial density of the branching dendrites tends to fill space in a constant density.
so how many synapses are there for a unit of space?
It varies but letās say 1 per cubic micrometer.
While we are at it - what is the density of the microcolumns and cell bodies?
| Tissue Property or Measurement | Value |
|---|---|
| Y, average interneuron distance. | 20.0 Ī¼m (1) (estimated) |
| P, average intercolumn distance. | 26.1 Ī¼m (1) |
| Ļ, slide neuronal density | 0.0013 neurons/Ī¼m2 (1) |
| l | 341 Ī¼m (1) |
| s, thickness of the thin slice | 30 Ī¼m (1) |
| radius or neurons (average) | 5 Ī¼m |
| % interneurons | 20% (2) |
| Model Parameter | Value |
| dn, interneuron distance. | 23.1 Ī¼m |
| dc, intercolumn distance. | 29 Ī¼m |
| Īø | Uniform random [0, 2Ļ] |
| Ļ | Uniform random, [0, Ļ/3] |
| % omitted neurons | 40% |
| Ī“dn | Gaussian distribution, Ļ=4.7Ī¼m |
| Ī“xn, Ī“zn | Uniform random, [ā6 Ī¼m, 6 Ī¼m] |
| Ī“xc, Ī“zc | Uniform random, [ā6 Ī¼m, 6Ī¼m] |
| N, number of images for average. | 500 |
How handy is that?
The microcolumns are on ~ 26-30 micrometer spacing.
Now we have to match that up with the ascending axonal bundle spacing that pokes through the dendrite arbors. You are looking at area 17 and the Hubel & Weisel paper has a lot to say about what goes on there.
so does this paper:
Organization of pyramidal neurons in area 17 of monkey visual cortex Alan Peters Claire Sethares
Sadly - behind a paywall.
The abstract is very helpful:
Abstract
In sections of area 17 of monkey visual cortex treated with an antibody to MAP2 the disposition of the cell bodies and dendrites of the neurons is readily visible. In such preparations, it is evident that the apical dendrites of the pyramidal cells of layer VI form fascicles that pass into layer IV, where most of them gradually taper and form their terminal tufts. In contrast, the apical dendrites of the smaller layer V pyramidal cells come together in a more regular fashion. They form clusters that pass through layer IV and into layer II/III where the apical dendrites of many of the pyramidal cells in that layer add to the clusters. In horizontal sections taken through the middle of layer IV, these clusters of apical dendrites are found to have an average centerātoācenter spacing of about 30 Ī¼m, and it is proposed that each cluster of apical dendrites represents the axis of a module of pyramidal cells that has a diameter of about 30 Ī¼m and contains about 142 neurons.
Is this typical for all areas of the cortex? Unfortunately, this useful paper is also behind a paywall.
The Organization of Pyramidal Cells in Area 18 of the Rhesus Monkey - Alan Peters, J. Manuel Cifuentes and Claire Sethares
Maybe you can hunt this down on your own. It has this handy histogram:
So - 20 to 30 micrometer spacing on the axonal bundles too. I suppose that makes sense that it matches up with the microcolumns. That 30-micrometer spacing seems to turn up a lot in these papers.
Like you I have to see pictures to fix this in my mind. 30 um out of maybe 500 um is a little less than 10 percent of the dendrite arbor field. I think it looks something like this:
Yes - before you jump on me - the dendrite arbors should be denser but the picture is too complicated to make out then.
@gmirey responded with this excellent paper:
https://www.ncbi.nlm.nih.gov/pubmed/15260960
and this one:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4370903/
Working with what we have so far:
Since we are visually oriented - a Graphic solution:
30 Āµm spacing of cell bodies
500 Āµm / 30 Āµm = 16.6
Rounding up = 17 = diameter of dendrite reach expressed in terms of cell bodies.
Set up a repeating field of cell bodies ā¦ inscribe a 500 Āµm circle
inspect and remove the cells x 4 corners in the local areaā¦
Any blue cell reaches the center of the inscribed circle.
Since this is a repeating pattern any starting point has the same solution.
The overlap of dendritic reach is:
(17 x 17) ā (4 x 13)
289 - 52 = 237 overlapping cells at any given point.
An inscribed area of 500 Āµm diameter circle gives an area of ĻrĀ²
3.141 * (250 * 250) = 196350 ĀµmĀ² = input area for any cell body
Assuming that the layer is I Āµm thick for a first approximation (prolly much bigger!)
Going with the 1 synapse per ĀµmĀ³ and dividing by the 237 overlapping fields
196350 ĀµmĀ² / 237 = 828 synapses per cell per 1 Āµm of thickness.
It is likely that the dendrites are spaced out through the thickness of the layer. It is very likely that this density is shared with all the layers. (2/3, 4, 6)
5 Likes
Yes, this exchange was very fruitful and @bitking insights are invaluable.
My model values differ slighly from those of last blue square, since Iām still attached to the hex lattice to layout even the microcolumnar structure, both from personal preference and from the belief that it will help quick-compute indexes on roughly circular areas to account for both those dendritic and axonal arbors concerns.
Iāll try to come with details on some of my latest figures with the hex lattice, and specifics of most of the layout, hopefully with a few drawings as wellā¦ sorry for the dense walls of text to this point. So thank you @bitking for those figures, discussion, infos, and now for bringing color to this topic 
Hi everyone. This time I also tried some visualizations.
First, meet the hex lattice
This here could very well be a top-down view of any cortical sheet model. Each microcolumn center would be identified with one of these points.
Each dot here has an equal spacing with any of its neighbors, and there are no ācorneringā neighbours distinct from āsideā neighbours, as would be the case with a square tiling scheme. I think it makes sense for an area so heavily concerned with topology as is V1. And in fact for any cortical area if we take the view that they all operate on same algorithm.
Now, we could object that using this weird layout seems to be lots of pain for little gain, however as shown below, there are very straightforward schemes to bind our common indexing for two-dimensional arrays to this:
You see that the grid above is still a common, regular, 2D addressable thing. Only skewed a little when representing it on display, or taking topology itself into account, so this layout has almost no impact on iteration times or global indexing complexity.
Now, an additional benefit is that any regular neighborhood is already a pretty good approximation of a circular area. Below is an illustration of an area which spans 4 neighbors from its center point, so this is a ā9-acrossā area:
I have a quick and dirty Excel sheet at my side, which tells me that there are 61 distinct points inside this shape. And it will give me that number for any of the āneighbours from centerā values. You can count them here if you wish.
With a few tweaks, you can fast-compute any offset-from the center (in the global addressing scheme described as a regular (x;y) index above) from any of these 0ā¦60 possible address values. And this holds true at any scale, thus increasing the speed at which we can compute the extent, retrieve a point address, or iterate upon any of these roughly circular areas, of any diameter.
So, an index taken as an āoffset-from-centerā in the figure above would fit in only 6 bits. But obviously, from the reflection and estimations we carried around together with @bitking, weād need to describe much larger areas for most of our concerns (which are both sides of the following matter : the extent of the axonal tree on one hand, and the dendritic extent on the other hand).
So letās try to describe the āmatterā in more detail.
You all know about the main excitatory cell of the cortical column : The pyramidal cell, and its HTM interpretation:
Letās add some rulers around this one guy, or one of its close cousinsā¦
Now that ~30Āµm for microcolumn-to-microcolumn center we settled on with @bitking is maybe not a figure everybody would agree on. It is on the lower end of the proposed scales from wikipedia (up to ~50 or even ~80Āµm). I believe however that, if one would take a greater figure than ~30Āµm, the philosophy of the proposed organization would not be much altered, and it would only reduce the count of stuff-per-area that we need to consider. So, ~30Āµm seems like the āworst caseā scenario.
Each soma of a pyramidal cell is set on one microcolumnar index, on the hex layout. Typical horizontal dendritic extent (whether basal or apical) is at its ease in a 8-neighbourhood (17-across), but dendritic plasticity would allow them to grow up to a 12-neighborhood (25-across) if really information-starved. My Excell sheet tells me that there are 217 distinct possibilities for the ātypicalā case, and up to 469 distinct possibilities when stretched. Those require at worse 9 bits to localize a subsegment relative to its cellās soma position.
We can thus fully encode a 3D position for the subsegment on 16 bits, if we take 7 more bits for the vertical, giving us 128 possible āinput templateā per cortical area. Now whatās an input template ? It is a description of the axonal side of the problem. These subsegments, as exemplified by the little part magnified in the blue box two pics above, are highly localized. Beside allowing the finer 8-coincidency threshold detection I propose, this high specificity of localization is the whole point of them.
An āinput templateā is localized in the vertical axis of the receiving sheet, and may allow several āsourcesā. A source is : a āsheetā of origin (same or other cortical area, or even a deeper part of the brain such as LGN), and a specific āpopulationā of cells within that sheet, sending axons towards the receiving area. Now, for each given source, there is in the model a precise topological relationship to the receiving area, which allow us to :
- define topologies as finely as the biological wiring we wish to simulate would require
- put an upper limit to the (otherwise huge) address of input cell, as viewed from a subsegment.
Weāll take a slice below to keep things simple in an example of topological mapping :
Maybe this looks like it is unecessarily complicated at this point, however now the wiring is completely specified. For a dendrite subsegment sampling the received input, the extent is applied the inverse scaling of the original mapping. In that example above, this would be divided by two, bringing the sampled neighbourood (around associated center) to two, ie. a 5-across area. You can check this on the above image, imagining how far apart on the source sheet could two cells having overlapping axonal arbors on the receiving sheet be situated. Any position in the original sheet layout thus gets a restricted area of incoming axonal overlap. For the (very lightweight) example above, it would look like this:
Now, the situation gets a little more complicated when the mapping is non-trivial, such as for ocular dominance stripes we have in V1, however I believe this is manageable.
Now, If we allow 12b or 13b per-synapse address, given a single sub-segment as proposed in my previous post, maybe there will be cases where the address-space is getting somewhat tight to allow multi-source with large axonal overlaps. Since Iām not confident we know of all weird-wiring cases which could potentially come up in the brain, letās say we increase that value to a very conservative 16b per synapseā¦ Iāll take whatever estimate you guys deem accurate for the max number of synapses in a subsegment having a length in the order of 40Āµm, but I believe this could be quite reasonable at this point.
At any rate, given a full-fledged ~6000 synapses cell, even with low filling-efficiency of subsegments at around 75% (this would bring the total synaptic āslotā count to ~8000), with 16b per synapse address and 4b for permanency value (permanency stored in another contiguous array, still very low profile as theyāll be stochastically updated), this makes the footprint of a cell at around 20KB.
The full connectivity state of a 100 thousand-cell simulation can thus be stored in around 2GB.
Seems high still ? Maybe. However the beauty of the topological structure is, no matter how many areas or minicolumns-per-area youād wish to reasonably simulate, those 16b-address per synapse-on-given-subsegment values would never change. Were it computationally anywhere in reach, a full scale brain simulation could thus, in my view, still operate with those same values.
Which seems reasonable enough to start opening Visual Studio and work from there for the V1 sim. Now I have some work on my hands 
7 Likes
Excellent breakdown and visualizations. One thing that pops out to me is a potential memory optimization from applying a strategy like this to the Spatial Pooling algorithm. Each input bit could have random potential synapses that are addressed in āaxonal arbourā coordinate space. Then given a target position in a destination sheet can become a reference point used to iterate the minicolumns which are referenced by those potential synapses. Think Iāll see if I can work this into HTM.go.
1 Like
Sure, HTM at the moment does not lean heavily towards topology concerns, thatās why we could maybe see this stuff mostly as an āextensionalā specification, which I believe could be integrated within current HTM Pooling or even TM algorithms, maybe with some addos for āt-1ā concerns (I think I will have to address t-1 also in the quite near future, maybe as what Iād call a distinct āpopulationā in the input template).
So, if you can get something running quickly along these lines and smoothly integrated with HTM that would be neat 
Yes, Iām also thinking on how to apply it to TM algorithm. One complexity (besides T-1) is the distinction of multiple segments per cell. From a certain perspective, that could just be an abstraction born from a lack of topology in the algorithm (as you pointed out, Jeff always describes the distal synapses as coincidence detectors, activated when several of them that are close to other in space and time are active). It may not be necessary to model distinct segments in an implementation of TM that includes topology.
Perhaps topology is not necessary for TM.
For my part my first ābetā in the simulation would be to consider that, besides position, dynamic input context, and output axonal wiring, nothing really distinguishes a pyramidal cell from another. So what must be true of a V1 L4 stellate, would be true of the cells which achieve TM. And Iāll try to fit-all in that scheme above.
Iām quite impatient of hearing back from your results, though