I’m not going to send an announcement for this meeting until later so everyone here on the forum has a chance to RSVP. I put a cap for attendees at 20 because I don’t want a huge group of people with questions. I would rather have you guys and gals from the forum to have a chance to bring something up.
I would greatly appreciate if you could reply here with any topics you want to ask about or discuss so I can give Jeff a bit of an indication on what the topics will be.
Question: Have you considered the location signal is macro in nature, originating at the map level from the somatosensory cortex as part of the vestibular/posture system?
In this scheme, the parietal lobe maps/areas between the various sensory cortex areas learn to map between the “where” in the body reference frame and the sensory reference frame(s). The “other end” of the vestibular/posture frame is the frame of allocentric reference; the support surface and the gravity reference of up and down.
The posture control system must predate the isocortex and must have always been present as the neocortex has developed from from the primitive precursor. I get this inspiration from reading the works of Dr. Cisek.
One could mount a substantial case that the original purpose of the parietal cortex was to build this allo-ego mapping and the temporal lobe remembers the locations of goal objects in an external world map.
That’s great Mark, thanks. Anyone else, if you have specific topics or questions you want to bring up with Subutai and Jeff next week, please post them here so I can prepare them a little bit.
If you are joining the Zoom meeting, I expect you will have a comment or question for Jeff and/or Subutai. If so, please post a comment below (or DM me) so I can let them read it beforehand. If you just want to watch live, you can catch this on my Twitch stream. I will also make it available on YouTube the next day.
For Jeff,
In the paper review “Navigating with grid and place cells in cluttered environments” I asked a follow up question about spatial processing vs spatial memory in the HC/EC complex. The answer that was relayed back sounded like we had a crossed message regarding the essential point that these are two separate aspects. There is no question that on some spatial tasks that involve spatial memory the HC/EC is vital; damage to the EC/HC disturbs tasks that involve forming a spatial memory. When the task is structured to tease out spatial processing separately from spatial memory it seems that the EC/HC is NOT required to do spatial processing.
You have to be very careful to look at what the task is testing.
Please see this paper. This quote sums up a key finding: “The findings suggest that the hippocampus is not needed to carry out the spatial computations needed for map reading and navigating from maps.”
I recall many other papers and videos where patients with EC/HC damage are walking (in labs, inside houses, and outdoors) and had no problems in normal navigation and object manipulation as long as the task did not require memorizing the details of navigation or spatial processing.
These considerations suggest that it is very important to separate spatial processing vs spatial memory in discussing grid cells in the EC/HC. I place spatial processing outside of the EC/HC and the formation of spatial memory in the EC/HC. I do not exclude the possibility that grid cells outside the EC/HC do spatial processing.
In recent work (such as what was covered in the COSYNE review session) there does seem to be grid activity outside the EC/HC and this could be where spatial processing is happening, but the episodic spatial memory forming portion seems to be centered in the EC/HC complex.
Let’s fast-forward in the near future. If you were to have access to a Brain-Computer Interface that has high temporal and spatial resolutions, resolutions that allow to zoom in and see separate neurons in any part of the human brain and possibly even stimulate them. Let’s assume, for the sake of this discussion, that it is a non-invasive device and it is completely safe, FDA approved etc.
What would be the first thing you would want to test with it? Do you have any specific thoughts on how it could help in further development of the HTM theory or possibly even in application of the theory in real life? What do you think are the first useful applications of such technology for both research and general public?
Question #2:
In the podcast with Lex Fridman Jeff mentioned an approach that he uses for creative thinking, which involves waking up at night and staying awake https://youtu.be/-EVqrDlAqYo?t=3729. Is it possible to elaborate on specific steps on how you do this? Is it somehow similar to what Thomas Edison did with hypnagogia and holding steel balls http://armchairdreamer.com/the-naps-of-thomas-edison/? In general I would assume that at Numenta there is lot of creative thinking involved. Is there any framework or techniques that you use to enhance creative thinking and to not be stuck with the same thoughts/ideas in a loop?
Question #3
What are your thoughts on lucid dreaming, dreaming where you are consciously aware that you are in a dream and where our prefrontal cortex seems to be partially awake? Specifically what could be the reason that we as species evolved to have this ability?
For Jeff: In the last research meeting, you’ve said the “location signal”(IDK if this is the right term) is egocentric at the lowest level(V1) and the output is more allocentric which feels like to me, implies the location signal gets more and more allocentric as it goes up the hierarchy. The question is, how is this “allocentrification” enforced? And is this also imply abstract concepts dwell in the higher levels of the hierarchy?
For Subutai: One of the goals of Numenta is to apply the principles you’ve found in the neocortex to machine learning systems. How could the sequence memory(or TM) be implemented into a deep learning system? Is it going to be almost the same algorithm but with differentiable operations or are you going to adapt it to fit better to a deep learning system like you did with the one from the paper “How Can We Be So Dense?”.
I would like to participate this next round, if there are any spots left. 5pm PST is 2am in Germany, where I am, but if I get a slot I would be happy to participate for a change. Your schedule does not make it easy for the european followers, but it is still a welcome opportunity.
A glance at this time frame matching may allow for better slots in the future. Unless you barely have participants from europe. I am a night owl by nature anyway.
I do not need to participate directly. I just read Mat’s comments and would be happy to follow the stream live. I will post any questions in this forum ahead of the event. Thanks for your open events! Very greatful in general to be part of your community.
What’s the next major test setting for TBT or HTM theory at large?
I’m mean the next engineering-embodiment – like the 3D object recognition task presented in the Columns paper. I hope I haven’t missed the answer to this somewhere!
If I’m right the Frameworks paper brings in the concept of Object Compositionality, and the role of Grid Cells & Displacement Cells to encode object behaviors. I think these components will prove paradigm-shifting for AI theory, enabling untold breakthroughs in application.
So I wonder: how do you envision demonstrating that!? Maybe some kind of expansion to the 3D object id task where the hand would feel an object behaving in its special way (like the stapler example), and learn to construct a new version of it from provided pieces? This would seem to require a system to understand both object behavior and compositionality. I don’t know. Would love to hear your thoughts!
I have decided not to post this meetup to Twitter. I think we have enough participation from this forum and good questions already to make more than enough discussion. Thanks everyone. There is still some space available because a couple folks dropped their RSVP because they did not actually have direct questions or comments they wanted to speak out. Remember I will be live-streaming this as well, so if you just want to watch, you don’t have to do anything.
A question for Jeff. I will not be online to hear the answer since it will be 3am in my time zone. So I hope that Jeff’s response, if any will be recorded.
Could each functional area in neocortex have shared grid cell modules say in the LGN or the entorhinal cortex? Each cortical column would multiplex with others to access this space.
This would solve several problems:
Each cortical column would not need to contain a full set of grid cell modules of its own.
Scaling of the sensor space (say vision) would be handled by just resizing the grid space according to the distance to the object being sensed.
Egocentric to allocentric conversions would be automatic since the metric space used would be the common grid cell modules which as in point 2 is resized according to the object being sensed.
Hi @Bitking. Is that follow-up question aimed at me or Jeff?
If to me, I really haven’t given that aspect much thought. I was just trying to solve a logical problem.
But evolution likes to reuse successful mechanism where possible, rather than reinvent functionality. If there are already grid cell modules nearby to cortical columns why add new ones to the columns?
As a guess, the path could have been, saccade muscle positions -> grid cells would then be reused for fixation position -> grid cells or something along those lines.
It was aimed at you but I wanted to flesh out the question a little.
So - the grid cells could be located remotely and the evolutionary innovation is a connecting fiber tract?
One of the “lowest cost” evolutionary sports is adding connections between areas.
I put this in the same “cost” as duplicating layers, areas/maps, or increasing the area of a region.
The Hippocampal complex, grid cells, place cells, etc… is used for spatial mapping or maps of cognition? Acording O’Keefe J, Nadel L., the latter seems to be more a map of the cortex than anything else (in the outside sensory world).
@vpuente what do you mean by the map of the cortex? For example is it something like a map that we have for the things in external world, but instead for our cortex. Meaning storage of locations of different parts of the cortex that are roughly speaking responsible for different functions. For example does it store where in the brain there is the visual cortex cortex, auditory cortex etc, like internal map?
