Thank you! This was very insightful. When you speak of L2 capacity, do you infer the number of different sequences L2 can potentially learn or the length of a single sequence? My concern is in saturation of inputs within one sequence.
As I understand it, pooling represents the union of all active cells within a sequence. Depending on the length of the sequence this has an upper limit of having all L4 cells being a representation of a very long sequence. That would make it impossible to tell one very long sequence from another long sequence. Is there a SP between L2 and the union of all active L4 cells in the given sequence?