Right now it seems like NuPIC has been used on problems where temporal locality is important, but not spatial locality. If I remember the "Hot Gym" example correctly, NuPIC was given power usage and date-time info and learned to determine how power would be used next. In that case, the added detail of how physically close the representation of date-time is to the representation of the number indicating power usage shouldn't matter.
However, with more spatial problems, like how to balance a robot limb, how one protein interacts with another protein, or how to find your car keys, the nearby inputs are much more important than the distant inputs. After all, you wouldn't start looking for your car keys by tracing your steps back all the way back to your very first birthday.
It makes sense that the local algorithms are only useful for a certain set of problems, but they are important problems. However, I believe it would be a good idea to optimize, or at least look for different ways of implementing things, because only being able to work on a 64x64 image limits being able to experiment with hierarchies on detailed spatial data. Meanwhile, here's a highly computational spatial operation done on a GPU:
That's a 2010 GPU simulating local interactions between a million particles at 2-3 FPS. If NuPIC is parallizable enough—and I believe it really should be, as long as 'local' is defined as a short enough range—then NuPIC should receive as much benefit from GPU optimization. Plus, because of how much of NuPIC is designed around the neural column, it should be easier to design it using a library for parallel or swarm computing.
For example, if I remember local inhibition correctly, the most activated columns in the spatial pooler inhibit nearby columns, so the highest activated remain activated, but less activated don't. That reminded me of edge detection, so I messed around with convolution matrices in gimp, and changed an edge detection matrix so that edges were highlighted within brighter regions and the inner parts were dimmed instead of removed:
What's interesting about the third image is that I can change the average brightness of the image by setting the central value between 24.0 and 25.0, and a pixel that was previously invisible in the source image is now hard to miss. Here's the central image for comparison:
That inhibition matrix can be made any size, and the central value will be between the number of items in the matrix and one less than the number of items in the matrix. After that matrix is applied across a spatial pooler of columns with, in this example, 24.5 as the central value (half original image brightness), the top 2% of columns could be chosen to maintain sparsity, which could also be optimized with a GPU.
Though, further researching led me to something called an unsharp mask, and it didn't produce the artifacts of the matrix I made. It highlights the 'invisible' pixel without generating any lines, so it should work for local inhibition without generating and detecting textures that aren't there.
Oh yeah, speaking of looking at libraries to use, I believe TensorFlow has a function for applying operations to individual numbers in an n-dimensional array of numbers and using the GPU to apply them. I think I'll look into using a 4-dimensional tensor to give an i,j location and receive a 2-dimensional "local connection strength" tensor centered upon the i,j location of a neural column and input matrix in separate 2-dimensional tensors. Then, to activate I'll apply rectified linear of input to each column, apply an unsharp mask to the activated columns, choose the top 2% of activated columns (within a certain area?), and then increase the "local connection strength" values for active inputs for activated columns and decrement the values for inactive inputs. Then I need to add boosting by computing active duty cycles over time, comparing them to local columns, and adding the a value related to the percent above/below the average inverted and added to one. That should implement a spatial pooler with localized connections, inhibition, learning, and boosting. (Sorry if this paragraph is extremely confusing. I'm using this to think out loud and keep track of everything I need to do later.)
I'll try implementing some of that tomorrow.