There is something i don’t understand about the htm concept, that is how the Inputs are related between different columns?
In the right side of the image, we can kind of assume there is only one way for the input to travel. But in the case of htm (thousand brain concept) the inputs need to act at specific columns to get the appropriate output. How does the incoming input act at specific columns (sensory columns <-> motor columns) apart from enormous columns that were present in the neocortex?
The cliff-notes version:
The classic view has problems with lateral connectivity. As the senses are processed the info maintains some form of topology. This is not as straight forward as it seems as the topology is in patches and does not match what you might expect if you were processing it as a simple visual map. That said, studies of actual wetware does not really support this hierarchy as you have diagrammed it. The “level of complexity” is supported but the bits and pieces are not “collected” into “grandma recognition” cells despite a few studies that seem to indicate that they do.
Turning to TBT, the presentation of a full cup in every column is a bit misleading. I asked Jeff about this in one of the online gatherings and he explained that this was not correct. Ever column sees whatever is presented to it by the portion of the sensory mechanism it is attached to. The lateral connections between columns allows columns to gather consensus that the fragment that the column is seeing is in agreement with the fragments the neighboring columns are seeing to “vote” on the most likely recognition of the possible matches that are stored in that column. The lateral connections between the various sensor streams expands this voting to achieve sensor fusion and better recognition.
I have my own interpretation of how this collects into object recognition but lacking the time to convert this into working code, it remains pure theory.
Why does it have to lateral, vs. in higher multi-modal association area?
If you want to structure the sensory stream into sensory & voting between sub units in a single layer you will be doing some version of lateral voting.
Well, that’s my question, why would you want to structure it that way? I see excitatory lateral connections as positional encoding by key-value pairs in a transformer, the weights / number of synapses per connection are trained vertically. Doesn’t have to be backprop, this training could be Hebbian, but it’s still vertical.
Q: does that work with very local connections?
This is very pertinent when considering cache memory or small memory for massive parallel computing.
Why not, Hebbian learning is local. And there are models that use apical dendrites as 5-8 layers of backprop. Unless you mean adjacent neurons, but I think those connections are inhibitory.
The local connections are both excitatory and inhibitory, with the lateral being excitatory.
Hmmm, what about lateral inhibition: Lateral inhibition - Wikipedia
I think that’s known as SP in HTM
Likely. We learn both the sensory pattern and accompanying states as a unitary learned local state.
Then again, with a local inhibitory field (basket cells) the lateral excitatory connections are sufficient.
I meant adjacent minicolumns, I guess it is excitatory within minicolumn, as redundancy for fault tolerance.
Yes, inhibitory connections are always through interneurons.