In about a year I will be writing my bachelor thesis. I have been following Numenta for quite a while now and I would be really interested in writing my bachelor thesis about HTMs. In this thesis I would not only want to discuss the current state of affairs in as far as HTMs are concerned but I would also like to tread new paths, meaning that I would like to do research, that is, research that would lie within the scope of AI bachelor student.
I would really like to hear your recommendations for a research topic that is relevant for Numenta and that would be manageable for an ambitious and high achieving AI bachelor student. A research question on the intersection between the Math/CS side and the neuroscientific side of HTM research would be really interesting.
Hello fellow bachelor student.
I’m as CS student in Taiwan; also going doing a graduation project about HTM this year. I’m (hopefully) implementing Sensor Motor Inference in HTM. Which, I’m pretty sure that I’m not going to nail the problem in one go.
Some projects that I have considered:
Sensor Motor Inference
Proper implementation of Temporal Pooling
Efficient Temporal Memory implementation on the GPU
Thank you all! The proposed research topics sound really interesting!
Marty, I’ve found some slides of Numenta about SMI, and also a 45 min video of Jeff Hawking talking about SMI. What are the information sources you are going to use for implmenting SMI?
Bitking, if I understand correctly, the hex-grid coding works on top of vanilla HTM? What is the current state of the technical details of this idea? For implementing this idea a theoretical framework is needed that has progressed enough to be almost ready to be literaly translated into code. Can you estimate how much research is needed before we get to this point where you can actually write code for this? Next year, I’ll have about 2 months to write my thesis(± 180). How much work will be needed on the theoretical framework before implementing is even possible. Can you estimate if this will be doable in the given amount of time?
Again, thank you both for your great ideas! Both ideas exactly match the kind of research topics I was looking for!
What would also be very interesting for me is seeing whether HTM Theory can help progress Deep Learning algorithms. For example one could investigate whether CNNs can be made (more) resistant to adversarial attacks by incorporating SDRs in CNNs for their robustness against noise. SDRs and Spatial pooling have great properties that can also be used in other domains of AI algorithm research.
And my previous work of abusing TemporalMemory and SpatialPooler as an universal function approximator. (Not uploaded to the forum. Its plan stupid simple)
Temporal Pooling is also an essential topic; as TemporalMemory has it’s limits and has to be expanded upon sometime in the fute. Ex: TM can’t learn the XOR problem without tricks.
Pieter,
My work is almost entirely based on simulations of the cells in the cortex and is completely unlike anything anyone else the people in this forum are doing. The test set, environment, and tool set are all in straight C and Perl. HTM happens to be one of the test configurations. Inputs are 2D maps of activation bits and maps of connections strengths, processing is all done on interactions of layers of maps, and the output is a map of bits.
How I do dendrites and synapses is somehow different than what anyone else here is doing.
All of the “standard” HTM configurations are based on OOP and are incompatible with what I am doing. Many people here struggle with making a simple encoder to process data sets. I am not ready to support teaching people how to transform data sets into the activation maps that I work with at this time.
Interpreting activation patterns as results is likely to be equally difficult for most HTM users.
So - I don’t have anything that you could build off of at this time. I am working on making the execution kernel and tool set a little less user hostile and writing more theory posts so users will be able to understand what the software is doing and how to use it. I have to admit that much of my work has been code first - comment later, and reading it won’t shed much light on what it is doing. I used to comment more but I keep changing the code as I try things out and the comments got to be totally misleading so I killed lots of then as the comments were actually counter-productive at one point. Such is the joy of trying to cram in a huge project in the few hours between dinner and bed each night. I would fire me if I coded like this at work.
If you are to the point where you could code an HTM from scratch then the theory of how hex-grid coding fits is at the same level as making a spatial pooler. If you don’t feel comfortable making a SP from scratch then this is not something that is likely to be helpful to you.
As far as integrating HTM and DL - you may want to read this post and give it some reflection:
This is what we are currently working on. Stay tuned, we’ll have a paper out on this soon. Specifically on how adding sparsity to DL networks via a SP hidden layer helps with noise and uses less computation.
