About capacity concerns.
Let’s say we consider a 2D patch of 2,000 cells, sparsely filled at 2% (40 of them active).
SDR maths, if we were to pick those 40 at random, account for a staggeringly large number of encoding possibilities.
Now if we add constraints relative to a grid
Each triangle of roughly 50 cells has only one ‘position’ possibility. That consideration already greatly reduces the number of distinct encodable concepts here. But… it’s still somewhat huge (50 to the 40th power)
Now indeed if each of those enforce its neighbors to align to the same position, the total number for the whole 2000 cells patch gets down to a ridiculous ‘50’.
But that’s not considering grid rotation. All those are envisageable. On a given 50-unit discrete equilateral triangle, number of distinct rotation schemes (when corner order is irrelevant) is at least 10. That rises our alphabet to a more satisfying ‘500’ already.
(More satisfying ? Yes it is not a staggering number… but it’s getting somewhat close to what I understand seems ‘satisfying’ from Numenta’s point of view for the number of concepts that a similar-sized patch processing SMI can play with).
Now, if we introduce back some amount of tolerance from perfect-geometrical grids, then per tile I wouldn’t be surprised that a rule-of-thumb ~7 different positions could at the very least qualify for that same grid. That would bring the number of concepts that our 2000-sized area plays with towards 500x(7^40), and we’re thus back towards the ‘staggeringly huge’ end of the matter.
And that’s considering that the griddy sheet is perfectly 2D. If there is some amount of depth to the grid-forming stuff, you can assume each unit in depth is to be multiplied to the value exponentiated above. Eg for a 4x2000 sheet, 4-units deep, your capacity is likely 500x(28^40)… [Edited… sorry for my poor math here]
(…Except we’d also be modifying the originally given sparsity considering this. Please note, BTW, that HTM-proposed 2% sparsity, giving rise to the “1 active cell per 50-cells triangle” figure, seems quite consistent with biological support for a ~0.5mm spaced Calvin-grid overlay on a 50µm-spaced cell-lattice… although there could be some debate on the 50µm spacing side of the matter).
Also, at some point, reasoning about the capacity of large patches has also to take into account that grid-forming is a general drive for the layer, but is not ‘ensured’ either… there is still potential for misalignment at the edge of two really disagreeing “bosses”
On large-scale models, that usage is indeed more biologically relevant if most such “large patches” are neighboring each other, as they would on a flattened view of neocortex.