I’m looking for some guidance from some more experienced people, and I’m hoping this is a good place for this. I’ve been following Numenta for quite a while, as I think the HTM method for learning robust feature representations is extremely promising. I’ve been using Python to help me play with and understand the algorithms, and I think I have a good understanding now. My issue is my Python, compsci, and math knowledge (or lack thereof) are beginning to hinder me now that I want to create a more robust implementation. Here’s what I’ve done so far (keep in mind, I’ve only worked on the SDR so far, and I’m hoping that solutions found here will help me implement the TM layer more easily as well):
I started with creating classes for neurons and synapses individually to understand how everything worked while following the documentation. I followed the BAMI outline here (https://numenta.com/assets/pdf/biological-and-machine-intelligence/BAMI-Complete.pdf) as best I could, but it was so slow I couldn’t test anything.
I moved to looking for a ready-made solution in one of the many HTM libraries that Google found for me. The problem here was two-fold: First, the majority of the functions were written in C++ (which I don’t know) so they’re really fast, but they’re not modular enough for me to use in the way I want. I want to be able to understand how the algorithms are actually implemented so I can tweak the parameters any way I want, or something like Pytorch where eager execution is fairly universal. It takes me some time to understand how things are working, so being able to just play with matrix objects really speeds up that learning time. Second, it took me ages to sort through all the different libraries and repositories to figure out what does what. There is so much jargon thrown around everywhere, it’s difficult to navigate if you don’t have experience in academic coding. So I abandoned this method because it didn’t seem like I would make much progress in what I actually wanted to do here.
I went back to my own implementation, and eventually moved to consolidating everything into numpy arrays, and using numpy functions in a creative way to get the output, perform the permanence updates, etc. This worked great at first, but I ran into a brick wall when I wanted to implement boosting and inhibition. Either there isn’t a way to perform the operations necessary in a fast manner, or I’m not knowledgeable enough to figure it out.
Finally, I started trying to work with sparse arrays. I have such a lack of confidence in my understanding of how this implementation should work, that it’s difficult to make any progress because I have no way of knowing if I’m moving in the right direction. And when I get done, there is no way of knowing if the python sparse array implementation will actually be any faster than the other methods I’ve used.
So as I mentioned at the start, I’m looking for some guidance on how I should move forward here. Ideally, I would love it if there was a discord server or something similar I could join to talk this sort of thing out. Is there an HTM library out there that has the sort of flexibility I’m looking for? Or perhaps someone knows of a good way to implement the algorithms with Python sparse arrays? I feel lost to the point that I feel like my best option is to try to go back to college for compsci, learn C++, then just come back and tackle the problem in 6 years.