RNNs and HTM

[watchdog]

Hi, i was wondering if it is possible to test and compare results between RNNs and HTM system.
I have never been envolved with RNNs and i am not sure if this is something that can be done.If anyone knows i would like to know

[rhyolight]

Well, we have the Numenta Anomaly Benchmark, which compares NuPIC to LSTM.

[subutai]

You can check this paper:

Cui, Y., Ahmad, S., and Hawkins, J. (2016). Continuous online sequence learning with an unsupervised neural network model. Neural Computation 28, 2474–2504. doi:10.1162/NECO_a_00893.

Numenta.com • Continuous Online Sequence Learning with an Unsupervised Neural...

[Deepak_Vellampalli]

Hi @subutai

Thanks for the link.The paper looked very interesting and I wanted to replicate the results. However, when I looked at the corresponding Github repository and tried to get it working, I failed.

I tried both the installation procedures shown in the picture:

When I tried by using pip install nupic htmresearch I got the errors shown in the next picture

And then I tried the ‘developer’ way, even it caused errors as shown in this picture:

Am very interested in this and any help will be highly appreciated

-Deepak

[rhyolight]

Run:

pip install nupic --user

What happens?

[Deepak_Vellampalli]

I already had NUPIC but I tried running the command again as per your suggestion and surprisingly it gave error. Please look at the picture:

[rhyolight]

Looks like a problem with the wacavtve package. Are you in that local directory when you install?

[Balladeer]

I’m pretty sure you are using Python 3 instead of Python 2. Python 2.6+ allowed both Exception as e and Exception, e, but Python 3 requires the the form Exception as e

[Deepak_Vellampalli]

@rhyolight Nope, am in home directory when I executed that command.

@Balladeer I think you are correct. Now I tried running pip2 install nupic --user It actually worked, but downgraded two of my packages which are python-dateutil and numpy. Look at this:

And then I tried the same logic for htmresearch. I ran pip2 install nupic htmresearch
Even that worked but gave similar error:

Any suggestions on this? Do you think it’s not a problem?

[subutai]

@Deepak_Vellampalli

Here’s another option. We try to put cleanly reproducible versions of our code in https://github.com/numenta/htmpapers but this particular paper is not there. It’s in the process of being cleaned up - the current version can be found on this branch: https://github.com/lscheinkman/htmpapers/tree/RES-609/neural_computation/continuous_online_sequence_learning_with_an_unsupervised_neural_network_model

There’s more info on how to run each experiment here: https://github.com/lscheinkman/htmpapers/blob/RES-609/neural_computation/continuous_online_sequence_learning_with_an_unsupervised_neural_network_model/DESCRIPTION.md

It still requires htmresearch, but there is a Dockerfile in the above branch. If you’re familiar with Docker, it can help alleviate the library issues you are having by running everything in an isolated container.

Hopefully we’ll merge this branch into the main htmpapers repo soon.

[Deepak_Vellampalli]

@subutai

Thank you so much for your reply. As per my previous comment, I couldn’t get htmresearch installed as pip couldn’t downgrade my simplejson package. I tried different ways to solve that issue but finally ended up executing sudo pip2 install nupic htmresearch --ignore-installed simplejson and it worked. It passed the test case too but not sure if it will cause any problems further(hopefully NO).

However, if I face any problems I will go with your suggestion of dockerization as it will give me a clean setup.

And also thanks for pointing out the htmpapers repository and also the current version of the project. Will make use of them for my work. I think community will be happy to see the cleaned implementation of this paper soon as this can serve as a powerful tool to prove the concept of HTM and its advantages over classic sequence prediction techniques.

Have a great day!

[Deepak_Vellampalli]

@subutai

Hello,

I have been working on those lines as I mentioned in my previous comment. I could install htmresearch successfully by ignoring few packages but it eventually failed in getting certain things working and ended up getting some issues regarding installation. Hence, I immediately switched to dockerization approach and started using https://github.com/lscheinkman/htmpapers/tree/RES-609/neural_computation/continuous_online_sequence_learning_with_an_unsupervised_neural_network_model as per your suggestion. After setting up, when I tried running continuous sequence experiment using python run_tm_model.py -d nyc_taxi, I got the following error:

I realized it’s because of the missing folder named ‘prediction’ which is available on htmresearch repository at https://github.com/numenta/htmresearch/tree/master/projects/sequence_prediction/continuous_sequence
Hence, I copied that particular directory into a docker container and ended up getting the following error:

I also got few other errors as well while reaching to this point. Does it mean the project is still unstable(at both the locations) and not ready to use?

Thanks,
Deepak.

[Deepak_Vellampalli]

@rhyolight @subutai

Any update on the working condition of this project?

Thanks,
Deepak.

[rhyolight]

Deepak, just to be clear, I’ll refer you to the bit we wrote about this repository a long time ago. I don’t want to bother the researchers to help the public run the code, but we still want the code out there. You can use it if you want, but don’t wait on us to improve the “working condition” of this project. This is experimental research code for the Numenta research team, so if it works for their needs we will not be changing anything.

I hope you understand.

[lscheinkman]

Hi @Deepak_Vellampalli,
I’ve fixed the missing directory issue and some OSX matplotlib warnings on this experiment. It should work now.
Try getting the latest from https://github.com/lscheinkman/htmpapers/tree/RES-609 and let me know if you are still seeing the problem.

Luiz

[Deepak_Vellampalli]

@rhyolight, I’m sorry for bothering you people on this again and again. As I tried the things that were suggested earlier by your researchers and they didn’t work, I replied to the same thread and expected a response. I neither meant to disturb your on-going research nor demanding your researchers to change the code for me or to help me in getting that code running. I think my intentions weren’t conveyed clearly!
And also thanks for making it clear that I should not wait to see the change in working condition of the project, that was helpful.

[Deepak_Vellampalli]

@lscheinkman Thank you very much for fixing the issues. I will definitely try them and see!

[subutai]

@Deepak_Vellampalli

I’m sorry for bothering you people on this again and again.

No worries - sometimes we need a little nudge

Also, thank you for finding and reporting the issues. It helps us make the results more easily reproducible.

[Deepak_Vellampalli]

I’m very glad to hear that! Thanks for making me feel more comfortable!