Hello, I saw references to that numenta implemented hierarchy back in the days but then abandoned the effort.
Is it possible to find details around this somewhere?
In my own implementation I have encountered a “natural barrier” for the number of layers that are practically usable and wanted to review other ways of Implementing them.
hi, I’m looking for the answer of the same question,too. The earlier version of Numenta’s HTM ,or ZETA1, there is both spatial and temporal hierarchy in it, however in the CLA, alghouth the SDR and hebbian learning of temporal sequence makes more sense than ZETA1, but it hasn’t incorporate the hierarchy in it yet.Currently what I’m trying to do is to combine the idea of both zeta1 and cla together.
I have implemented hierarchy in a way that is consistent with my theoretical understanding of the brain, and works really well with HTM without any changes to it. However I am running into the issue that after a number of “regions” are connected it does not make any more sense to add more layers.
There seems to be a “natural barrier” to how many regional layers are useful. As the rate of change through the hierarchy decreases exponentially for every layer.
Extremely deep hierarchies are probably only useful with abilities such as generating motor commands. Based on the degree of complexity of every region in the neocortex, there are definitely some things to add.
I think you wouldn’t encounter that natural barrier as earlier for a more complex environment.
What kind of mechanisms did you use for your hierarchy?