Hey @azza, sorry for the wait.
I can’t say precisely how best to do this, but I’ll try and clarify the concept and point you toward an example of it applied.
Conceptually the idea of ‘temporal pooling’/‘union pooling’ is to recognize the sequence itself, by monitoring the behavior of other TM regions.
For instance let’s say you encounter a familiar sequence:
A,B,C…X,Y,Z
When input C arrives at timestep 3, a normal TM region (that has seen this pattern before) just recognizes C in the context of A,B, and predicts D. A pooling region however, recognizes that this is the English alphabet.
So while the activity of the normal TM is constantly changing every time step (which bits are active), the pooling layer should change more slowly. Theoretically the pooling layer - once it recognizes the familiar sequence - should stay constant throughout the sequence (up to letter Z in this case).
I don’t know the inner working of this as applied in current research code, but here’s a discussion thread discussing it in detail including network schematics: 2D Object Recognition Project - #104 by Zbysekz (schematic on post 100).
I’d also recommend checking out this Numenta paper, which demos 3D (virtual) object recognition. Again I don’t know exactly how it’s done, but I know that the information from separate sensors (TMs) is aggregated in some way to classify the object.
In this case the different sensors are different fingers, each feeling different parts of the objects, and thus gathering different information on them. Since each sensor has limited info, it can’t narrow down the set of possible objects nearly as fast as when info is shared among sensors.
Classic example is a coffee cup object. One finger is on the round handle, so it thinks the object may be a ball – but another finger is on the lip of the cup, so the sensor can eliminate the possibility of a ball. When the different sensors’ info is shared the object is recognized much faster.
Anyone please correct me if I’m wrong/missing anything, but I think that’s a core finding from the paper:
Luckily the code used in the paper’s experiments is also available in research repo:
Hope this helps!