Good news, everyone.
I’ve reproduced the results of this paper. It is implemented as a lightly modified Nupic Spatial Pooler.
It sort of works, not as well as the original, but IMO well enough to show that the core principals are sound and worthy of further investigation.
These core principals are twofold:
- Each mini-columns proximal excitement is put through a low-pass filter, which induces stability. My best explanation for this is that the excitement reacts slower than its inputs are changing which forces the grid cells (read: SP mini-columns) to learn large contiguous areas of input.
- The filtered excitement has a fatigue, which shapes the SP’s receptive fields into spheres. The fatigue also uses a low-pass filter. The fatigue slowly reduces the excitement over time, which effectively limits the amount of time which a grid cell can be active for. The competition then causes these spheres to be packed into the environment.
Both of these effects happen at the cellular level.
I have posted a more complete write up at:
In the future I hope to experiment with using a low-pass filter to induce stability in the output layer of Numenta’s two layer model.
Dmac