Hmm this is interesting, although you will get stuck as Jeff did, when you try to find way how this could work with repeating grid cell module. You would have to have wired up all neurons in all combinations. That seems to be not the generic way, because it should be all over the cortex. And stating that the output of this network is learned, doesn’t help either.
The second part that Jeff et al. were fundametally right about is the combination of several GCM’s to get very large spaces, so they are not throwing everything. That is also very magical. As i said, i am not very confident at this topic right now. But it makes me calm, that i am not the only one who struggles.
Ever heard about Stephen Wolframs A New Kind of Science ? I received it as gift for Chrismas (my wish 
) (but it is online free available too) and i am impressed. His first basic motivation was exactly this: Rule 30. In other words, how simple programs can lead to very complex behavior. Counter intuitive at first, but true. It touches very lot of things, from fibonacci numbers, prime numbers … up to Computational irreducibility. BTW he now makes a giant step towards fundamental Theory of Physics Wolfram Physics Project. Joining theory of relativity with quantum world.