I have tried to understand the differences between the spatial grouper and the memory of sequences, and where the concept of (union) is applied
2 Likes
They are 2 different pieces of functionality. The Spatial Pooler is a feature extractor. And the Temporal Memory is a sequence memory.
The working on Temporal Memory heavily relies on the input SDR have a fixed sparsity (at least being sparse). Yet the input SDR may not be sparse or contains redundant information. The Spatial Pooler removes them and generates a SDR that a TM can use.
The union operation is just for proofing the properties of SDR (the principle we use to perform classification and detect anomaly)
4 Likes
In the biology - the union property only applies within a single dendrite and it’s response to different input perceptions. The union part extracts similar features of different perceptions on that very small part of the cortical sheet.
This is important in a very subtle way - the math of SDRs shows that it is highly unlikely for two different SDRs to match so the trained pattern of synapses would almost never get an exact match. Put plainly - maybe this cell would only fire if it saw the exact pattern of the skin of a melon oriented a certain way at a certain distance. This specificity would not be very helpful and would be extremely wasteful of the neurons.
Instead, they can learn to extract common features of many presentations and generalize some microfeature that is common to these presentations, at this location in the cortical sheet; sort of like a puzzle piece.
This results in a cortical column knowing many shapes and orientations with high specificity; many possible puzzle pieces at this location. Using the lateral connections as described in the “thousand brains theory” this micro-features NEXT TO a different micro-feature builds to part of an object recognition; the puzzle pieces that are next to each other snap together when the corresponding features match.
Since this is temporal memory, stepping through the sequence of perceiving of some object generates transitions of different clusters of micro-features. This acts as a filter to reject spurious object recognition and quickly latch onto the sequences that match in all related columns. There are many possible pairs of micro-features but this object drives ALL pairs to activate matching lateral connections over some larger area of cortex to be recognized as the motion progresses.
As the sequence progresses, each micro-feature sequence at each cortical location is different - it is a distributed representation. They all cooperate to recognize the input, each doing its own little bit and voting with lateral connections. It does not “come together” at some higher level, it starts and stays distributed as processing continues through various stages.
Everything is connected: This area of recognition will result in driving the thalamus into tonic mode and result in spreading of patches of activation to surrounding cortical maps. The L2/3 connections between this map and other maps activated by the thalamus will carry the recognized pattern at this portion of the cortex to other maps for further processing. This process spreads as each map in turn continues the process, all the way to the temporal lobe & the EC/HC complex to register as experience.
6 Likes