We have not hosted any community events in awhile, so I think it would be nice for Jeff and I to have a recorded chat when I’m in the office next where I represent the community and Jeff answers my questions. This won’t be a live stream like an office hour, I’ll be editing it and posting it later. So you won’t have a chance to ask questions live.
I will be talking with him about some technical aspects of the current research, the direction Numenta is heading, and future plans.
That being said, are there any specific topics you would like us to talk about?
Yes, a ton actually. Let me post just some of them, and if the format is ok I will post more.
On page 71, Jeff talks about how conscious effort can change the way patterns are retrieved. Do we know how the conscious effort is generated in the brain? Intuitively, something needs to happen to break the circuit of input sequences and predictions happening in the brain.
I am still not sure about the difference between temporal and spatial patterns. Seems like all patterns coming in are just temporal, with spatial configuration encoded in every frame. What benefit does this separation serve?
Seems like, in V1, each cell encodes a particular small feature. Do we know what mechanism is responsible for associating the feature with a cell? (page 112)
How does each cell work with both upward sensory flow and downward predictions? It can’t be like a domino effect where you kick the dominos from both ends because a cell can only take input and generate one spike as the output. Does it mean that the outputs from higher layers connect to lower level cells’ inputs? Basically, can the prediction flow excite or inhibit a cell?
I really enjoy it when Jeff gets into the guts of the biology. Any discussion on how HTM algorithms map to the cortical layers would be very fascinating (i.e which layers interact for temporal memory and temporal pooling, and how they interact). Maybe any thoughts on when/if Numenta will be trying for multi-region HTM models - towards the hierarchy.
I would be interested in Jeff’s thoughts on the functional relevance of neural activity at the fine temporal scale. The timesteps of HTM can be envisioned as demarcated by an underlying inhibitory oscillation, but the global lock enforced by discrete timesteps is a strong assumption. For one, it precludes the ability of different cortical regions to dynamically couple in and out of phase as necessary (the communication-through-coherence hypothesis), and it also constrains the ability of the network to adapt to varying extrinsic rhythms in the body and the world. There’s also mounting evidence that conduction delays might serve an important computational purpose.
In other words, what role do you see for fine-grained spiking and oscillatory dynamics in a mature theory of the brain?
The neo cortex has a lot of cortical area’s, like V1, V2, … , MT for the visual system. These area’s communicate with each other, the output of V1 goes to V2, etc. But what is known about the communication between area’s? Are the connections between area’s the same for most area’s or are they always different? Can you see the direction of the communication? Do other area’s notice when lots of columns are bursting in the connected area? Etc.
Question: What is embodiment, that a machine might have some?
(probably don’t need to unpack that as the implications should be fairly obvious)
many thanks for putting this forward to Jeff.
That is a very broad question, I would like you to unpack it a bit. I assume you want to know more about sensory-motor integration? Or is this a more philosophical topic?
I would be interested in any thoughts about timing (WRT sensory-motor integration). While less important for sequence memory and streaming data (abstracted by implementations that rely on discrete time), timing can be extremely important when interacting with the world (depending on the use case, of course)
Yes, I think so. @jakebruce question seems to hit on a somewhat broader topic (to be honest, I am not at the same level of comprehension as most other members on the forum here). Just wanted to be sure to point out an interest specifically in how timing relates to current thoughts on sensory-motor integration.
Thanks Matt. Yes, this is the sensorimotor question(s) by and large. So am more than happy to roll-in under that heading. Very keen to learn more about your new advancements to HTM core theory.
Where that might be extended is: a human body per se is not necessarily the only definition of “embodiment”. Jeff has, I think, articulated that “anything” that can move through a perceptive field and learn about the structures of that field is essentially exhibiting (cortical) intelligence. Thanks.
I can confirm that. A thing with intelligence does not have to have a physical embodiment to traverse spatial input over time. You can think of a web-crawler as a perfect example.
I would like to ask about new findings or theories that deal with the process of segregation of processing into different sub-compartments(color blobs in v1, strips in v2, and output to various areas, etc) and different output streams. For example the process of arealization, where not necessarily identical output is sent to the various areas an area connects to, likely leading to area specialization, any new findings or theories regards the level of genetic predetermination of between area connectivity versus algorithmic self organization processes?
It sounds like you are talking about the organization of cortical circuits occurring on a level that is currently higher up than current HTM implementations. We are still focused on what is occurring within one layer of cortex. We’re not yet modeling interactions between different regions, so I’m not sure he will have anything to say about that at this point.
EDIT: now that I think about it, this question fits into discussions about cortical columns, so we will fit something about this in. Thank you.
