Hi Numenta and community! I’m a grad student working on HTM interested in making a novel contribution to the algorithm, and I thought I’d seek feedback from those who know best to help determine what openings there are and what angles I could take. One particular issue I’m interested in is that of handling multiple input streams. In the video ‘Multiple Fields in NuPic’, Subutai mentions the current limitation of the algorithm to handle only as many as 5 streams or so.
I’d like to eventually be able to feed any number of data streams into the algorithm, for it to determine which fields play important roles in what circumstances and in what combinations. My basic intuition about this issue in how its handled by humans is that we, when given a large number of input streams, would choose to focus in on some subset based on what we know to be important, and the more experience we have the more we come to distinguish which fields and combinations thereof have what meanings in what contexts.
It seems to me that the current method of handling multiple fields, of feeding the SDR’s representing each field all into one HTM region (if I understand this correctly) does not scale well since the SDR’s eventually become saturated with ON bits and they lose their sparsity (hence the limit of ~5 fields in the current implementation). So I have the thought of giving each field its own region, and then using hierarchy in some way to learn the interactions between the different fields in different scenarios as they unfold over time.
I thought for instance that if there were 3 input fields, each could get its own region on the first level on the hierarchy. Then the next level up could have 2 regions, the first forming from the cumulative outputs of regions 1 and 2 from from level 1, and the second from the cumulative outputs of regions 2 and 3 from level 1. Finally there could be another level of hierarchy (level 3) with a single region, formed from the cumulative inputs of regions 1 and 2 from level 2.
Whatever the exact producer, I simply hope to achieve the ability to feed in any number of inputs fields to the algorithm, such that the only limit to its capacity would be the amount of data. Of course it would take more data to find patterns of patterns of patterns between numerous input fields than from a single field, though I hope to be only limited by the supply and quality of data, without worrying about any ceilings in the capacity of the algorithm itself. I’m happy to go into more detail on how the outputs of 2 regions from hierarchy-level 1 would be mapped to a single layer in hierarchy-level 2, though I don’t want this post to get too too long.
In ‘On Intelligence’ I remember Jeff explaining hierarchy by describing how it allows information coming in from multiple different senses to give more predictive power than either sense could on its own, and that there was some level in the cortical hierarchy where the predictions from different senses (say sight and sound) are combined. So in theory it seems that hierarchy may be a plausible way to handle the issue of multiple fields, Needless to say I’m very curious for any thoughts anyone had on this issue. Thanks so much !
– Sam Heiserman