This morning at 10:15AM PST.
Jeff Hawkins will present something. I don’t know what it is yet.
Visual Cortex:
Directional orientation
Spatial patterns
Scale and speed of signal interpolation by columns/minicolumns peaks closer to input sense source ?
[10:50am PST]
[Jeff focused on diagram in bottom center of whiteboard]
Florian: [mumble] to break them apart and [mumble] to put them back together [simultaneous crosstalk from Subatai and Jeff].
Did anyone catch those [mumble]'s ?
[11:00am PST]
Florian mentioned “theta” power fluctuations.
Theta waves? Excitatory and Inhibitory waves of neuronal activation ? Where and how?
1 Like
Question:
Has anyone tried visualizing HTM networks using UMAP? It seems to me that UMAP’s topology-preserving dimensionality reduction would be a very effective way to visualize grid cell modules and basis vectors in an HTM network.
1 Like
I noticed some properties of how the spatial pooling algorithm interacts with SDR properties a while ago that seem to naturally encourage grid cell -like structures in the cortex. It also suggests various “phase transitions” between dimensionalities of grid cell modules (1D -> 2D -> 3D, etc), as well as place cells (0D grid cells?). It also gives some interesting ways of thinking about how HTM actually organizes receptive fields in order to model inputs, and provides some nice connections to both how humans reason in spaces (introducing landmarks as an important idea), as well as perhaps some connections to attractors from dynamical systems.
I figured it might be obvious enough that Numenta would have come across it already, but after watching some of the recent streams, I’m starting to think that might not be the case.
I’m a bit busy so it’ll maybe be a while before I can get a full explanation written up, but I’m thinking I should probably make a thread about it. It might give everyone at Numenta a new tool for understanding HTM.
Sound interesting?
2 Likes
I’ll read it!
1 Like
I’ve been thinking for the past couple of weeks that a mini-column might be capable of representing a scalar value. Mostly I was thinking of this in the context of storing a scalar coefficient that could be used to reconstruct an input signal as a linear combination of basis functions. It occurred to me that there was sufficient resolution in even a sparse representation in a small number of bits (~100) to form effective approximations to a wide variety of input patterns if they were used to essentially scale each of the basis functions at a somewhat coarse granularity. It might even be that some mini-columns utilize a non-linear scale (e.g. logarithmic) to represent a wider range of values.
2 Likes
If the synapses are a network of connections between cells, conveying activation by pulse trains, what would it mean to have a scalar value?
The synaptic action is to depress the potential in the vicinity of the the host dendrite, and in concert with enough neighboring synapses, to trigger an action potential wave in the host nerve cell.
More or less connection strength changes the "shape’ of the network but the ‘value’ conveyed is the leading edge (phase) and repetition rate (strength) of the pulse train. Both are essentially binary signals with no additional analog component.
Where would a scalar value enter into this signalling chain?
Perhaps scalar value is not the right concept. I was thinking of basis functions in a mathematical sense and their composition in the typical linear fashion. However in the context of mini-columns, these might be indices corresponding to a specific learned filter[^1] under different contexts (i.e. lighting, scale, color-variation, etc.). Each of these contexts could involve discrete or continuous variations on a particular theme (the theme being the pattern stored in the filter). The ability of an agent to prospect about sensory input that it has never experienced before could be due to the compositional nature of these filters using the MC representation to evoke specific contexts (or variations thereof) for each filter independently.
[^1]: Filter in this context would correspond to a set of learned (in the Hebbian sense) connections to the neuronal axons within the receptive field of a given mini-column.
1 Like
But what means dimensionality in the auditory cortex? Can be those close mini-columns “related” by the timing and not by “spatial” patterns?
Why not both timing and spatial pattern?
HTM is all about sequence.
But a sequence of what?
Spatial patterns!
The formants of speech are a pattern of sounds - chords of frequencies detected as sensations by hair cells in the inner ear.
These sensations are transmitted to the auditory cortex, which is arranged in a tonotopically pattern, where the location in the patch corresponds to the pitch of the tone.
These sounds of speech can be arranged in a spatial pattern:
The transition between these points forms words.(Sequence)
More on this general process here:
Possibly the best book on the subject:
In lecture format:
You should be able to buy a copy for under $5 if you look around. Amazon charges too much for this.
5 Likes
For audition, there are a number of different dimensions in which you could expand an incoming signal. The frequency domain would be an obvious choice. Each of the nerves attached to those hairs in the fluid of the inner ear is probably encoding different frequency responses to the incoming signal. Let each one of these responses be a axis in some high-dimensional frequency space. Then your grid cell modules are encoding the amplitudes of the frequency responses.
That’s likely an oversimplification. If audition is anything like vision, then the encoding is probably taking place at a slightly higher level. The actual encoding space probably involves integrating several spike trains from these inner ear nerve cells simultaneously. A spatial pooling type of process then learns commonly occuring input patterns among these spike trains and then trains a bank of filters that can be used to efficiently encode an input signal as an SDR in temporal memory. This would explain why one can hear ones name mentioned in a crowded noisy environment and attend to it immediately. The specific sequence of frequency patterns that is your name is probably one of the most well developed (and likely among the first) filters that each of us has.
2 Likes
It can be … spatial by the means of the “binaural” bands (like ocular bands do in V1 for 3D).
But inside each band (there are many mini-columns), there is only subcortical inputs. Seem like just “temporal” correlation between the dominant frequencies is more logical.
Don’t think that at AC1 level, there is nothing about sentences, words, or even phonemes. For example, seems quite hard to achieve there the speaker invariant output.
2 Likes
These higher level organizations of information is where the multi-map communications come into play.
It is not all done on a single 2D map. Binorality (phase delay) , timbre, pitch, volume, sequence, and rhythm are all features that are to be extracted at a lower level and recombined for higher level pattern extraction.
If you look at the Auditory Cortex you can see that there are many intermap connections.
This follow the general pattern in the visual chain where “simple” features are extracted close to the sensory patches, and progressively more complex combinations of these features can be found as you ascend the hierarchy. As these features become more abstract it gets difficult to relate back to the original stimulus as the response patterns no longer seems to be directly related to the sensory stream where the sensation are received; the inter-dimensional mapping is not clear.
The abstract codes the brain uses do not seem to be organized the same way as the ones we use to encode things in the sensory realm. It is in some high-dimensional space where humans have no direct experience or intuitive understanding. This property defies our usual attempts at symbol decomposition into content.
This is why understanding the theoretical underpinnings of the H of HTM is so important.
I personally feel that the theoretical tools to understanding these high-dimensional codes this will be found in some extension of set theory.
2 Likes
I agree.
But, the problem is way before the H. How the relationship between close mini-columns is established. In other words, what is the purpose of L6->L4 modulatory inputs? Is some form of “spatial” reference? or its just sensory context? (i,e, the representation of close sensors activity).
How do Grid Cells […] get anchored to sensed objects?
The paper (Kropff & Treves, 2008) has a good and biologically constrained theory of how this might happen.
https://onlinelibrary.wiley.com/doi/abs/10.1002/hipo.20520
* I highly recommend this paper. Don’t let the title of it discourage you; the title is a pun, a joke which you will understand after reading it.
3 Likes
This is the heart of both my hex-grid concept and Numenta’s thousand brains theory.
The assumption in both cases is a co-operative voting method to join local fragments of representation into recognition of larger spatial/temporal pattern.
This action is bidirectional so that the joint action acts as a filter or de-ambiguator.
Applied to a larger section of cortex this filtering action acts to both pull signals out of noise and generalize.
It seems not applicable in 3D, according to experimental pieces of evidence.
this is a related discussion regarding the 2-D vs 3-D nature of grid cell representation here:
I am not really surprised that an inherently 2D sheet preferentially maps 2D space.
1 Like