I think these cell types are probably all pyramidal cells doing different things in different layers of cortex. They are really talking more about column / layer architecture than cell types with this misleading label. Or am I wrong?
Here are my current questions (I have partial answers to some questions, will present my final findings next week):
- What quilting patterns exist in what areas?
- What is common about the features they are projecting across the cortex?
- Which ones are continuous and which have abrupt borders?
- What are the relative sizes of quilting patterns?
- Why do some patterns create more distinctly separated areas than others?
- How could lateral connectivity support or create these quilts?
- What is the capacity of a pyramidal neuron to support these quilts? How many could exist in the same region of cortex? (are they even enforced by pyramidal neurons?)
- How are these quilt groups related to grid cells and locations?
- How do quilting patterns affect the RF of different layers? (why does it matter?)
Also, here is a partial list of the papers Iâm reading that inform this subject:
- The columnar organization of the neocortex - V B Mountcastle
- Demonstration of Discrete Place-Defined Columns in the Cat CI - Favorov OV
- Laminar differences in the receptive field properties of cells in cat primary visual cortex - Gilbert CD
- Functional architecture of the macaque monkey visual cortex - D. H. Hubel, T. N. Wiesel
- Orientation Selectivity and the Arrangement of Horizontal Connections in the Tree Shrew Striate Cortex - Bosking WH
- Visual maps in the adult primate cerebral cortex - Rosa MG
Any thoughts?
2 Likes
This is making sense to me. Every time you apply this quilting, representations get more general.
1 Like
This is way over my head, so I hope Iâm not putting my foot where it doesnât belong. But seeing that Peter Schiller presentation discussing the Hubel and Wiesel experiment (behavior of neurones in the visual cortex of anaesthesised cats (at 37:36)), I remembered something.
By coĂŻncidence @jhawkins talks about that same experiment during the interview he gave on the Braininspired podcast. He explains that the results from anaesthetised cats is vastly different from data with animals that are awake. It starts at 39:56. According to him, those findings are long since overruled.
4 Likes
Yes and no.
If the animal is awake and attending you have the counter-flowing stream arriving at the perceptual area doing what HTM calls the location signal. This additional signal is a confounding factor is trying to sort out what is flowing UP the sensory stream.
The H&W work and what has followed is useful in sorting out what is going up the hierarchy.
3 Likes
These cats were not doing object recognition. I think the responses they were seeing were rote edge detection from the retinas sparking and echoing through the cortex, but nothing was catching them.
2 Likes
Iâll be leading a journal club on Crazy Quilting next week with the Numenta research team. I will record it and show you all, so stay tuned for that. In the meantime, here are my notes. Iâm not promising they will make any sense.
3 Likes
Very good start. I am planning to hunt down the reference(s?) that mentions cells OTHER than pyramidal in mapping RFs. I canât remember which paper(s) it was in so I will have to get lucky in search terms.
This is one of the problems in having 13K papers in the library, with many of them scans. Naming and key words are everything it finding anything. Unfortunately - I did not learn this until it had grown to a large mess.
I have some housekeeping to do; a lot of it.
Sigh.
3 Likes
Well, I think I would also see different color patterns if they gave me a double dose of ketamine. ;-).
2 Likes
@rhyolight - looking at your notes - how are you feeling about the possibility that the scope of a quilt dovetails to the local spread of activation fields?
Also - How are you feeling about the âwell shuffledâ part where the local features in the association area include a mix of local and distant sensing interspersed with other modalities? I can see how some of this would happen with level skipping between the maps and patches in each hierarchy.
It seems to me that these will have to be in the same local area to form an SDR that includes this information to do sensory fusion. If it is not done all in one space it will have to be done scattered over some map-2-map connections. Considering the known topology of the inter-map connections it seems likely that much of this fusion is done in the association areas.
or
https://www.cell.com/neuron/pdf/S0896-6273(15)00765-5.pdf
or
and
https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(13)00221-0
2 Likes
I think of cell types as including things like which layer the cell is in and what other layers or brain areas it sends signals to. Pyramidal cells with different connection patterns can be intermingled, so I think theyâre referring to what you are describing but not necessarily separate physical layers.
The way complex cells are defined seems to vary a lot (like different math or whatnot). They seem to mean anything which isnât a simple cell. It all seems too vision-specific to me, so I would be wary of specializations.
The same cell type can be simple or complex depending on the visual submodality. In layer 6 of macaque V1, there are three types of corticothalamic cells which roughly correspond to three types of LGN cells (parvocellular, magnocellular, and koniocellular). The ones which correspond to parvocellular-like LGN cells are simple, whereas the other two kinds are complex. [1]
The barrel cortex seems like the best region for determining whether all regions have quilting patterns, which I understand to mean mapping onto the cortical sheet with discontinuities and generally what is or looks like disorganization.
Barrel cortex has a very orderly columnar organization in layer 4. There is one cortical column for each whisker, called a barrel in L4, and this columnar organization is present in some but not all other layers or sublayers. Things like whisker bend direction could be mapped though, which could have quilting patterns.
If you need a quilting pattern for the map of sensory patches/whiskers, it might have to be in the space between those columns, called septa in layer 4. Septa is driven by a different subnucleus of VPM than the barrels [2].
Whiskers are discontinuous, so each barrel column has a discontinuous border with its neighbors, except there are cells between those columns with less obvious columnar organization or no columns.
Are coordinate transformations relevant to quilting patterns?
I think the septa in barrel cortex and the koniocellular system in primate visual cortex are involved in coordinate transformations or sensory anchoring, involving dual-projecting L6 CT cells. When a rodent whisks (rhythmically moves its whiskers at e.g. 20 hz), there is a map of the whisked space in L2/3 [3], which is proof that primary cortex cares about location in some way. I only have preliminary evidence that septa is involved in this but a bunch of things point towards it.
You mention distance in your notes. One possibility is that septa determines distance by comparing info from different whiskers. Cells in the septa have larger receptive fields than those in the barrels, so they could compare whiskers. Iâm pretty sure it has something to do with behavior or location though since cells vertically aligned with the septa and a couple related parts of the thalamus are closely associated with motor cortex.
I can list some sources if any of this is relevant and give sources for what I didnât cite.
[1] Parallel Processing in the Corticogeniculate Pathway of the Macaque Monkey - PMC
[2] http://www.jneurosci.org/content/29/13/4089
[3] https://www.cell.com/neuron/fulltext/S0896-6273(17)30352-5?code=cell-site
I donât know exactly what you mean by this so maybe this isnât relevant. (I should go find where it was mentioned on this forum elsewhere.)
Part of POm (higher order thalamus for barrel cortex) receives a direct sensory input (and input from L5 of barrel cortex), so I guess level skipping can include the sensory input. Itâs a different sensory input (trigeminal nucleus) than barrel cortex L4 gets, with larger receptive fields.
2 Likes
Iâm not so sure of that anymore. There are two types of quilts, the first is the major sensory field projection onto the cortical substrate. It is globally discontinuous, but locally continuous. This is a very different type of quilt than the âfingerprintâ / âzebra stripeâ quilts. Iâm not sure these patterns encourage the spread of activations any more than a direct projection of their positions within the sensory space would.
This only happens in the âmajor sensory projectionâ described above, specifically in somatic cortex, but also to a lesser extend in visual cortex, as there is folding of the visual field along V1/V2 border and further discontinuities further up the hierarchy. And here I would not call the input âwell shuffledâ really. Most of the sensory area patches are in generally the right place. It just looks like a drunk threw them together⌠or that the nerve bundles just sortof fell together that way as they joined up toward the spinal column.
Maybe the more important function of this minor quilting (fingerprints, stripes) is to create local groups so that computations can occur. It reminds me of how mini-column neighborhoods are necessary during spatial pooling when there is topology in the structure. Cells need to be grouped somehow for local computations to start happening, and I think this discontinuity (especially where borders between slabs are sharp) might arise to allow it. (I donât know how to describe the more continuous minor quilting, like orientation and RA/SA yet.)
2 Likes
I look forward to the considerations that will come from wider sharing in your journaling group.
BTW: that is a wonderful Idea and I am considering how I may be able to add that to my life with my local contacts.
2 Likes
This refers to the type of inter-areal connections between maps.
Look at figure 2 in this paper:
https://www.cell.com/trends/cognitive-sciences/pdf/S1364-6613(13)00122-8.pdf
The drawing labeled âCombinatorial learning and emergence of semantic categoriesâ is a perfect example of this level skipping concept. This happens all over the brain. The older ideas of hierarchy envisioned an orderly progression from senses to the association areas. Modern mapping techniques of tracts have moved on to connection matrices that show that the level of connections between two defined areas.
It is almost possible to say that everything is hooked to everything to some degree. Some areas more - some less. Several posts earlier in this same thread I posted links to papers that explore this same topic in great detail.
@rhyolight I donât think I have done a very good job of explaining what I mean when I say âshuffling of receptive fields.â
I hope this diagram will show how topology is preserved but mixed with expanded scales of receptive fields.
I intend that these are maps at different levels of a hierarchy. It is implied that the ovals in the green box are a sample space for individual SDRs.
Considering the way that connection matrices work in the cortex - in a general sense this could be any path from map to map to map.
4 Likes
I donât know much about this topic, but Iâm skeptical of connectivity matrices. Leveling skipping is definitely a thing, still. I just get annoyed with connectivity averages because I focus on one cell type at a time and connectivity is really important.
The cell types and subcellular target (e.g. proximal or distal basal or apical apex) matter. An orderly hierarchy and dense connectivity (in terms of which region targets which regions) arenât contradictory. You could probably tell distal/proximal synapses apart by comparing results from retrograde tracing and responses evoked by the presynaptic cells, so connectivity matrices are pretty useful in that way.
For an unrealistic example of why an orderly hierarchy and non-hierarchical connections are compatible:
Think of a hierarchy of spatial poolers for a few different senses, each with a temporal memory. Predictive connections onto distal dendrites between each temporal memory donât need to obey the hierarchy and can go between the hierarchies for different senses, and it is still a strict hierarchy in terms of minicolumn states.
I agree that this idea can be a bit disturbing on first exposure. It certainly was for me.
I have been reading through the results of mapping tracts using newer imaging technology and it really does seem as if there are more connections than were originally found using injection tracing.
On the functional side of things this does make some sense. In a computer program you can access a variable from any part of the program that needs a sample of whatever information it is conveying. The brain does not work that way - if information must be available it will have to be conveyed by projections from one place to another. For sensor fusion I can see a very real need for taps of various parts of different sensory streams to be connected at different levels of processing.
2 Likes
Some preliminary findingsâŚ
For each of three cortical areas I investigated (somatic, visual, auditory):
- there is one primary sensory field projection:
- transforms sensory input field onto physical cortical substrate
- direct input from sense through thalamus
- exists at birth
- structure / topology does not change with learning
- globally discontinuous, locally continuous
- there are (many?) minor input property projections for each sensory input:
- allows different input sources to express at same point in sensory field
- does not exist at birth, emerges with learning
- these typically look like stripes between binary variables
- may have distinct (occular dominance) or fuzzy (SA/RA) borders
- could emerge because of direct input source differences or because of the first level of pattern recognition in cortical processing is responding to specific low-level input patterns (like SA / RA)
- low level input patterns
- observed only in anesthatized animals in V1 (as far as I could find)
- orientations of a line or direction of movement in V1
For #1 above, visual and auditory cortex projections are most similar. Both have mirrored continuous sensory space projections. The somatic cortex also has mirror representations, but the way it treats sensory field location seems different. It is not a continuous field, but a series of semi-discontinuous patches of space, each with local continuity.
For #2 above, when input is from different physical sources (like two eyes or two ears), there is a physical merge of neuron fibers into a striped pattern (for V1 you can even see the stripes in the LGN). Eyes and ears seem to project directly to grouped sections of neurons with sharp borders. The SA/RA stripes in somatic cortex are similar because there seems to be a small amount of cortical processing occurring in order to respond only to one stimulus or the other (note that SA/RA stripes donât have the sharp borders of the binocular / binaural stripes). This is learned over time, I donât really know how they emerge, but I also donât think it matters too much as long as each input can project to the right location in the sensory space. For skin, there is not the problem of separated input sources (both RA/SA can be deduced from the same patch of skin, the neurons firing are physically close to each other in the sensory space, not in separate eyes/ears). Why there are even SA/RA stripes, I donât understand.
For #3 above, I think this is an echo of sensory input into an unconscious cortex. Once we can monitor a waking cortex, these transformations will likely make a lot more sense wrt what is in the visual field. It is hard to say anything about these patterns except that there is likely some very low-level pattern recognition going on even in the receptive fields of the first level of input to the cortex.
3 Likes
Some of the early work on #2 âlocalâ patterns it that they are learned on exposure to the environment. Deprived upbringing results in impairment past the plastic period; this is the Gabor filters that are formed by learning.
On the local stripes - these form stereo perception in the eye and phase/location signals in the ear.
Considering that these are strong location signals I would think that this is very important to the current grid thinking that is being developed at the Numenta mothership.
1 Like
In any experiments where the animal is anesthetized (most all of them above), the object location signal doesnât exist because it is generated in the cortex. As the eyes / ears focus on objects in the sensory field, the job of the cortical columns will be the same no matter. There is no experimental data on this, as far as I can tell.
1 Like
Iâm not sure what you require in the form of experimental data. I have been looking at many different papers over the years and there is considerable work out there.
This is a quick grab of papers with no sorting but I have dozens to pick from.
2 Likes