Thanks for your thoughts. I agree with a lot of what you said. I think HTM will require input from multiple sensors in space to really shine on hard tasks like object modeling. We need to split up the senses so we have different cortical columns modeling different locations in space, attached to different sensors. That is how sharing lateral connections will enable sensor fusion. We think this is how local patches of one sense are fused, and we also think this will apply across sensor boundaries.
But one thing we can do with today’s tested theory is see how it can be applied to the Deep Learning systems of today to improve performance or accuracy. That is currently what we are looking into, and I have to say that from what I’ve seen so far I am encouraged that we can make some impact on the current Weak AI scene by adding some HTM ideas.