Just wanted to say that you gave a very good answer to your own question. Also, to prevent confusion of other readers, I think you meant active columns rather than neurons.
@jakebruce The video and approach seems a very good fit for HTM. Lately I was thinking about a visual sensor that just captures the edges (I can access environment geometry) or color change in visual data to sparsify the input. The RGB color sensor I am using at the moment has fixed sparsity but it is not sparse at all and I think I am crippling HTM because of that. Columns need to map to a very large subset of the input bits. A sparse visual sensor became one of the priorities for the agent at the moment. I can detect the changes in intensity as well as discussed above. Thanks for the direction.
I am looking for the simplest starting point. Would I be good if I just turned on bits that changed intensity value by a threshold? Also, do we apply inhibition to the neighboring pixels? If not, why not?