Hello again, I’m back with more questions related to coding my own HTM implementation in pure Python3. I’ve gotten the spatial pooling algorithm part done at this point, but I’m having some issues with optimization. I’m hoping someone can help me figure out how best way to handle the sparse distribution of synapses during the update and overlap phase. I’m currently using numpy arrays for everything, and I’ve tried 2 different methods so far:
I initially tried using np.random.choice() with a static seed for each minicolumn (where the seed is the index of the minicolumn) on the input array to extract an input array for each individual minicolumn. This seems to be very memory-intensive and CPU-intensive due to the np.random.choice() method. I considered using a mapping array to map only certain indices of the input array to each minicolumn, but that still seems like it would be very slow.
I switched to using a binary mask that gets generated for each neuron that gets applied to the permanence matrix after updating each time. This is much faster than the above method, but it seems to be completely counter-productive to the idea of sparse connections in the first place - to reduce computation cost. With this method, I have connected, active, input, and permanence arrays for each neuron that are ALL the same size as the initial input array, so it eats tons of memory.
So how do I handle this issue? Am I thinking about this the wrong way? I tried looking at the Nupic code, but I can’t make any sense of it. Is it better to split up the input array for each minicolumn? If so, what is the fastest method for doing this through each iteration?