I want to start thinking about building robotics applications where HTM and the thousand brains architecture are applied. One way to do that is to try to build something simple. So here are some ideas about building some that does object recognition and is simple enough to explain and demo.
Some up-front assumptions:
- you don’t need a robot, a simulated world will do
- there must sensors in an environment
- either the sensors must move, or the environment must change
- if the sensor moves, the movement must be predictable, or it must be encoded as input
- sensory input is topologically mapped into a topological arrangement of cortical columns (CCs)
- each sensor will have multiple CCs
- in each CC, a pooling layer will attempt to resolve objects above a sensory layer (SP/TM)
- CCs will have lateral distal connections to share between object layers
So now, the hard questions we have to answer:
- do we require directed movement? Can random movements suffice?
- if not, the HTM must generate the movement command, and I’m not even sure who has even thought about how to do this
- how do we know when the sensor has moved from one object to another object (or a sub-object)?
- How can we take a movement command and encode into something meaningful in object space?
- Can we classify objects based on a collection of object layers representations?
- Can we create a hierarchy across the sensors?
What are the first steps? I am honestly tempted to start thinking in 2D spaces. I’ve seen some of @mrcslws’s experiments where there is a grid, and each spot in the grid can have a feature there. Movements are simple because you can restrict them to . We could start with one “sensor” that can sense one spot in the grid and get the features there. It can move, and when it does, the movement is encoded into distal input to the sensory layer of the CC, which should predict what features will be there (if it has seen any). We could hard-code some movement directives, such as affinity to explore and find more features and see if it starts predicting the right features as it explores the space.