I was reading (and watching videos) about TBT sensori-motor inference on object recognition, it all makes great sense…especially the roles of long range connections between columns, which makes inter-column voting, broadcasting, possible, further enabing integration and cooperation of multi-modal sensory inputs. For example, we can see something (involving visual cortex columns) and then describe it in words (involving language area cortex columns).
Then a thought occurred to me: what if the recognized object is not that famous Numenta coffee cup, but a simple hand-written symbol, for example, “3”?
It could be just me hallucinating, or it could be the case that Hawkins and colleagues actually already solved the “symbol representation problem in a connectionist neural network” without yet being aware of it – he might deserve a Nobel prize if that’s the case, because it could be the “one small step for a man, one big step for the mankind” kind of breakthrough.
Hallucination or realization, here goes the reasoning:
A column in the auditory cortex (Column A) recognizes the sound of “three”. (After training/learning, of course)
Another column in the visual cortex (Column V) recoginzes the image of a hand-written character “3”,
Long range cortex connections from either/both of the above columns activate another column somewhere else in the PFC (Column C). Let’s ASSUME this column represents the abstract concept of “3”, a count number in preschool math. It is invariant with regard to the following various kind of visual inputs in (analogous to the output of a temporal pooler being invariant with regard to various location-sensory input SDR sequences when touching THAT coffee cup):
A column in visual cortex (Column V_a) that recognizes three apples;
A column in visual cortex (Column V_b) that recognizes three lego pieces;
A column in visual cortex (Column V_c) that recognizes three coffee cups;
Column C in PFC is connected to Columns V_[a,b,c] through long range axons. C represent the abstraction/generalization of the semantic meaning of “3”, Columns V_[a,b,c] represent the concrete examples/instantiations/intuitions of the abstract concept “3”.
So altogether, Columns A, V, C, V_a,V_b,V_c makes symbol “3” work … as a recognized “object”, as symbol, that represents a meaning.
I highly suspect these multiple CC interactions/integrations are how the wetware in our brains handles symbols (and languages, and reasoning …). I feel the mechanisms in TBT so far are almost capable enough. Replace the concept of “3” with “grandmother”, we might be able to view the “grandmother cell” debate with new perspectives, Numenta TBT style … or am I imagining too much?