Neurons’ “antennae” are unexpectedly active in neural computation

There was this found in their model that indicates “spikes initiated within local dendritic regions” traveling both ways through their respective dendritic tree:

Our simulations show that sub-threshold PSPs from the distal dendritic regions of the On-Off DSGC are heavily attenuated by propagation to the soma, but that spikes initiated within local dendritic regions can propagate with high probability to the soma and back-propagate to the remainder of the dendritic tree. Therefore active amplification of DS appears to take place during spike initiation in the dendrites.

This seems close to what you are describing. If inhibition is optionally included then back-propagation can be stopped.

1 Like

Great, thanks Gary. But it’s weird that back-propagation is to remainder of the tree. It implies that adjacent but not spike-causing synapses are potentiated. Fire together, wire together would be reinforcing the spike-causing ones.

2 Likes

A weird idea keeps coming to me - that is that the fire-together/wire-together thing is partly or wholly a local thing based at the synapse level. I envision a mechanism where the products of the metabolism of firing interact with the extra-cellular chemicals to promote growth.

I extend this with the thought that some residue of this experience (a metabolic residue) makes the synapse susceptible to the soma based firing wave as the local back-prop training wave as envisioned by Numenta.

3 Likes

This new RNA related information just arrived by email from Camp:

We are now at a level of detail where things like that need to be modeled in.

So don’t stop there! Keep going on that thought. Almost everyone reading this is probably wondering “back-prop training wave what”? Best I can do is recommend asking Bitking for more details on what he knows about how that part works.

The predictive model assumes that some part of the dendrite arbor fires on a sensed pattern, making a prediction that we will fire again soon. The prediction is a partial depolarization but not a firing of the main soma body. This prediction is the central theme of the Temporal in HTM.
Later (but not much later) the main soma body is fired - a very big event on the cellular level, and the action potential fans out through through the output axon AND back up the dendrite tree. (The back propagation wave) HTM theory has an assumption that the synapses that predicted this firing are strengthened; this is a key element of the theory.

4 Likes

Thanks Bitking, that made easy sense of everything. The paper mentioned back-propagation but exactly how this related to HTM theory was best explained by you.

I’m now wondering why there is no online HTM neuron in a dish, lab, yet. Brush it a certain way with activity and soma fires. Does this sound like a relatively easy one to you too?

1 Like

Sorry, I can’t keep a cactus alive.

1 Like

After adding a supply of virtual microglial cells to nibble the buds just right we’re likely into 10,000 hours of work to get almost there, but we have to start somewhere.

Thousand Brains type thinking simplifies the problem by being possible to use metabolic network level SDR’s and whatever else is needed from HTM theory to approximate overall synaptic and RNA behavior that leaves metabolic residue or whatever. All by itself a single neuron is expected to have a small but impressive brain of its own that makes good predictions, but where it’s for that reason expected to not be easy to model it would be a shame if it were easy as an electronic logic gate. At least requires no watering at all.

TBT depends on this older HTM working as described; this is assumed as being in place.

As far as glial cells not covering synapses - that just makes sense. If they are to be exposed to axons to make contact it would not work very well if the were covered up.

With all theories being to some extent “tentative” it should at this early point in time be no problem to where need be take “HTM” right out of TBT. You may have already seen my somewhat related reply to Matt for the last hangout.

There only needs to somewhere in the cortical mass be found the whole bunch of smaller brains with a moving straw views of the outside world. In fact I was quietly hoping that the theory would end up needing to include at least some of the individual neurons. In that case TBT only becomes even more awesome.

I wouldn’t say this at all. The sequence memory algorithm we explained in our 2011 white paper is essential to all further Numenta theory. If we find that this model is inaccurate, we will have to rethink all the sensorimotor theory and TBT.

2 Likes

Yes, you can “rethink” your theory. That was more or less what I was saying. The whole thing does not necessarily have to be thrown out just because one part needs updating.

From my perspective TBT would be a shame to abandon now. For that reason Bitking’s mention of “TBT depends on this older HTM working as described” was a little bit alarming to me.

1 Like

This is the first in-vivo evidence for dendritic spikes, right? That can be a big thing.

3 Likes

It is one thing to adjust details of HTM sequence memory as we better understand biology.

It is another thing to throw the whole mechanism out because it is not working at all like we thought.

I have not seen any evidence of this 2nd thing yet, has anyone else? Even this paper that started this thread seems to confirm it.

1 Like

@rhyolight I don’t think that it is the issue - it’s just that the connection to the older papers get more tenuous as time goes forward and there is a large conceptual distance between today’s TBT and the papers it is based on.

I am always grounded in these older papers and the connections though to todays work but it does take a certain depth of understanding of the theory to see these connections all the way from the synapse mechanism level through the mini-column level all the way up to the impact on the higher level representation level.

This does go further, to the map level representation, onto to the map-to-map level, and global interactions between the maps and the sub-cortical structures. I have been working at these levels for many years so for me; theory tweaks at different levels always have to be considered through the lens of what impact it has on all these levels.

While this can be complicated it also has the advantage that known behaviors at various levels puts some constraints on possible answers at other levels. This sanity check does give a nice go-nogo check on proposals when I encounter them.

This does make it very hard to find someone to talk at a peer-to-peer level at parties.

3 Likes

I think I know what you are saying. For me the still biologically unknown HTM related details (but fair amount on cell intelligence that now exists) makes it so that when I look for evidence of a direct connection I end up back at complex cells already having some ability to piece together a moving straw view of (depending on what they are best adapted for sensing) what is of most interest to them in the outside world, which in turn gets expressed by column level behavior. TBT thinking this way holds true, but I cannot use HTM as further evidence to support it. And if that’s true then the problem is that the HTM model for a neuron has to have at least the inherent behavior of a slime mold then to be most useful to biology need cell to cell RNA exchange communication into well networked germ cell line chemistry.

I see a safety net for what Jeff described with a straw and most defines TBT thinking, which is not dependent on the fate of HTM. With that considered what you described seems more like the need for something that takes a fresh start, like a HTM-2 reboot that features the best of the old school HTM School videos.

Once you add the action of the lateral axonal connections to the basic HTM model you get the powerful features of the TBT or Calvin tiles.

There is no need to go beyond that to mechanisms like RNA exchange.

Yes, each cell in the mini-column has a very restricted view of the world but it shares that view with the neighboring cells using these lateral connections.

3 Likes

Times like this I start thinking about how nice it would be for you to have a full HTM neuron in a dish type visual of all that working.

The forum sees interest in using HTM for genetic applications, so I’m not alone in most wanting that sort of thing someday included. The ultimate would probably be a database connected model that molecular biologist would all want to code the behavior of their discoveries into. But don’t overly mind my sometimes thinking way ahead. Years from now after what is still a mystery has been explained there is at least that left to work on, to help keep the forum active working on something, never know.

RNA also just happens to be one of main origin of life interests, connecting through multiple sciences then finally to neuroscience. Whenever it seemed to me that HTM theory alone was not enough to support TBT there was my faith in that, to see me through. Although in my approach you have to endure my need to connect TBT to as much biological accuracy as possible, like metabolics and especially make sure to explain where any antennae are. Our from different approaches arriving at the same conclusion is better than one alone.

It’s excellent that we are fully on the same page, my view exactly. It’s why I indicated something that the cell has, influencing behavior column level behavior. Problem is in finding a fast easy way to model how all this going on inside the cell, to people who think we’re both nuts to think that way. What ever happened to the old school few rows of circles with lines to all others where magic Weight formula does all the rest?

This is exactly the point where you connect to cell chemistry then are next into RNA behavior anyway. To help I will include text I compiled from what I wrote for Reddit. It covers current planetary chemistry origin of life theory on up to our brain level, neuroscience. The vital RNA World OOL molecules are narrowed down to small number in this summary:


https://www.nature.com/articles/d41586-018-05098-w

Where you start here DNA becomes a came after thing that happens on its own by having RNA behavior right, not something coded in. This means there is a way to skip all the DNA related complications, for those looking to apply HTM to origins related genetics of some kind. Once you let them know about that, HTM is already into the most exciting RNA science around. In this case it’s real easy to get to the very starting point, where all begins. This is what I have so far to go with it:

As opposed to being like an alive on its own substance DNA is an inert material with properties of a crystal, RNA molecules use to “etch in stone” for future generations. Folding and unfolding packages it up for replication. Exact pattern reflects path taken through geologic time, what its RNA creators learned, instead of (as with snowflakes) exact path through atmosphere.

You may have earlier seen this that shows I had no problem finding all of the (things to look for) requirements for intelligence in self-replicating RNA:

Clues to the origin of intelligent living things are found in rudimentary molecular systems such as self-replicating RNA. Since these are single macromolecules that can self-learn they are more precisely examples of “Unimolecular Level Intelligence”, as opposed to “Molecular Level Intelligence”, which may contain millions of molecules all working together as one.

REQUIREMENT #1 of 4 - SOMETHING TO CONTROL

The ribonucleotide sequences are a memory system that also acts as its body. The motor muscles of RNA are molecular actuators, which use the force of molecular attraction to grab and release other molecules. The catalytic ability (chemically reacts with other molecules without itself changing to a new molecular species) of ribonucleotide (A,G,C,U) bases combine to form useful molecular machinery. Where these bases are properly combined into strands they become a mobile molecule that can control/catalyze other molecules in their environment and each other, including using each other as a template to induce each others replication. Unlike RNA that exists inside a protective cell membrane (as our cells have) these RNA’s are more directly influenced by the planetary environment, which they would have once have been free to control. Modern examples include single strand (ssRNA) viruses that can control the internal environment of their host and may now have protective shells with sensors on the outside for detecting other suitable host cells to enter and control, for the purpose of reproduction. In some cases after invading a host cell other sensors can detect when conditions are right to simultaneously reproduce, thereby overwhelming the immune system of their hosts, which could otherwise detect then destroy them.

REQUIREMENT #2 of 4 – SENSORY ADDRESSED MEMORY

On it are molecular sites, which can interact with nearby molecules to produce repeatable movements/actions. Its shape can include hairpin bends that are sensitive to the chemical environment, which in turn changes its action responses to nearby molecules and to each other.

A variety of properly ordered molecular species can easily be produced by wind/water motion or wet/dry cycles, resulting in quadrillions of different combinations all being tried in all the environments where the stuff of life in great quantities constantly accumulates, such as deep basins and via “skimming” onto ocean shorelines. Their combined activity also changes their molecular environment, much the same way as living things have over time changed the atmosphere and chemistry of our planet.

REQUIREMENT #3 of 4 - CONFIDENCE TO GAUGE FAILURE AND SUCCESS

Molecular species that can successfully coexist with others in the population and the environmental changes that they caused are successful responses that remain in the population. Molecular species that fail are soon replaced by another more successful (best guess) response. The overall process must result in collective actions/reactions that efficiently use and recycle the resources available to multiple molecular species or else there is an unsustainable chemical reaction, which ends when the reactants have consumed each other, resulting in an environmental crash.

REQUIREMENT #4 of 4 - ABILITY TO TAKE A GUESS

For a rapidly replicating molecule RNA editing type mechanisms can become a significant source of guesses. Also, molecular affinity will favor assimilation of complimentary ribonucleotides but where some are in limited abundance another ribonucleotide may replace what was previously used. The change may work equally well, or better, for their descendants.

Doing the same for DNA results in describing what the RNA and support molecules cause to happen in a fur ball that on its own does nothing. DNA is then more precisely a long term memory system that to RNA’s would be analogous to a library or internet, a transcribable storage for what has been learned through time by ancestors except for RNA is billions of years older and can be directly transcribed into unique RNA individuals that serve a vital specialized purpose in the complex molecular society of a living cell.

Behaviorally speaking there is a fractal unfolding that begins with RNA behavior. There is then a cell that has antennae and such like multicellular animals do. Unlike DNA alone it’s easy for the system to meet all circuit requirements. After that is unfolding the same to multicellular behavior level.

After replication DNA genome 3D cross linking and chromosome territory intermingling returns back to the same places. For the most part RNA controls the self-assembly and self-disassembly of the crystal lattice structure. Surrounding membranes that self-assemble around the DNA nucleus further help hold its shape.

In this reply I showed what to expect of a DNA alone (by tangling like I did or more controlled) cross linking it would when wet be strong enough to stay together on its own, not bowl of overcooked spaghetti that falls apart upon contact with water.

After chemical extraction from the rest of the cell DNA is most like a crystalline skeletal remain. In it are the imprints of RNA behavior that uses it to sustain a living system through time.

1 Like

News that indicates cells are more animal like than most people expected is all good for a theory that requires a thousand times more brain power from each neuron.

If there is a 2nd thing (in addition) going on then it’s most like what my long time favorite on Cell Intelligence has been experimenting with, mostly pertaining to stem cell behavior:

http://www.basic.northwestern.edu/g-buehler/FRAME.HTM

It’s (at least currently) beyond the purpose of HTM theory to answer questions at that level of detail, but it does not hurt to have some evidence for stem cells having a brain of their own too.

Along with that is of course the additional surprise that the origin of the RNA programmer of DNA has been well enough pieced together to start in chemistry, towards a more complete neural model that includes genetic behavior.


Even for supercomputers an atom by atom molecular dynamics simulation is too much of a number cruncher. For the complex RNA’s (and their associated molecular subunits usually made of protein) the important detail produces more like virtual critters, where it can be useful to tweak behavior by making them intelligent enough to adjust their own behavioral parameters to match experimental results, based on RNA nucleotide and/or protein sequence and folding environment.

In my opinion the Thousand Brains thinking and biologically relevant HTM neuron makes Numenta an early pioneer of the emerging science of Cognitive Biology by specializing in human brain level cognition. Your work is able to influence the work of those who get more into the chemistry level details.

2 Likes

I also found something in the previously mentioned Cell Intelligence information that seems to relate to the limitation mentioned in the Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in Neocortex paper that reads:

One limitation of the HTM model presented in this paper is that it does not deal with the specific timing of sequences. An interesting direction for future research therefore is to connect these two levels of modeling, i.e., to create biophysically detailed models that operate at the level of a complete layer of cells. Some progress is reported in Billaudelle and Ahmad (2015), but there remains much to do on this front.

First there is this:

Chapter 2 will use the apparent symmetry and identity between the branches of the phagokinetic tracks of dividing cells (an example is shown below) to argue that cells are programmed to measure angles and time durations.

Then Chapter 2 concludes:

Not only are the tracks of sister cells related, but the shapes and internal architecture of their bodies (i.e. their cytoskeleton) appear as symmetrical or identical as their tracks are [See ref 1,ref 2]. This suggests that the programs that determine the future movements of the cells are implemented by building and re-building the inner architecture of the cells [See ref 3].

A HTM neuron already has paramaters that approximate chemistry related cytoskeleton architecture changes. It would be a task of adding in an approximation of angle and duration behavior of self-motion. In social amoeba experiments time durations are by a almost all soma quickly learned, to ahead of time prepare for an air blast they are sensitive to.

This got me to thinking how placing a cell in a neural environment can make outside world stimuli directly applied to its antennae would become the new self-motion it senses. In this way a cell could become uaware it has become stationary, only sense a like through straw like view of itself moving around in the outside world “we” the cell colony call “us” are through our eyes seeing. A HTM example would be sensing the moving up and down with energy consumption, while the cells make predictions useful for preparing for what happens to them next.

1 Like