Install NuPIC using Docker

One of the biggest challenges of getting NuPIC up and running is managing dependencies. By using Docker, we hope to streamline this as much as possible, and eliminate dependency hell. Docker is a virtualization method which leverages Linux containers (LXC) to create lightweight and portable environments for running code with all of the required dependencies. The nice thing about Docker is that it allows us to manage all the dependencies of the core NuPIC code with a Git-like interface. So, as the code base evolves and dependencies change, you can update the environmental dependencies in a similar way that you update your local code base.

Because Docker uses Linux containers, Docker will only run on Linux. However, other operating systems can create virtual machines to host Linux, which in turn will host the Docker container. We will show the steps for setting up NuPIC on Ubuntu and OSX below. Docker is only officially supported on Ubuntu, but Docker should also work on the following Linux distributions: Arch, Gentoo, Fedora, CentOS, and Amazon Linux (AWS).

Allan Costa, a member of the NuPIC community built a Dockerfile which we are currently using.

Install Docker

Furthermore, there is a Vagrant configuration with instructions at https://github.com/numenta/nupic/tree/master/coreos-vagrant for running CoreOS, a linux distribution well-suited for running Docker containers. If you already have virtualbox and vagrant setup, this may be a good option for you.

For the commands below, you will need to enter these through your SSH client.

Pull the NuPIC Docker image

sudo docker pull numenta/nupic

Test it out

Run the Python unit tests:

docker run numenta/nupic python /usr/local/src/nupic/scripts/run_nupic_tests.py -u --coverage

Run the Python HTM Network API tests:

docker run numenta/nupic bash /usr/local/src/nupic/bin/py_region_test

You can similarly run the other tests, but it will fail the Python swarming tests, unless you install MySQL in your container.

Run the hotgym example:

sudo docker run -i -t numenta/nupic /bin/bash -c "python /usr/local/src/nupic/examples/opf/clients/hotgym/simple/hotgym.py"
sudo docker run -i -t numenta/nupic /bin/bash -c "python /usr/local/src/nupic/scripts/run_opf_experiment.py /usr/local/src/nupic/examples/opf/experiments/multistep/hotgym/"

To access the Docker container via shell:

sudo docker run -i -t numenta/nupic /bin/bash

After running the command above, you will be inside the Docker container running numenta/nupic. You can navigate around, and edit the NuPIC code, which is located at /usr/local/src/nupic. To exit the Docker shell, type exit.

The NuPIC Trusted Builds are continuously updated as NuPIC is updated, so it’s likely that your running container is already up to date. Should your running container fall behind, you can build NuPIC at any time following the usual build steps. Because the NuPIC Dockerfile builds completely from source, all of the necessary build tools are available and environment variables are set.

cd $NUPIC
git pull origin master
python setup.py install

Build Your Own Docker Image

If you would like to build your own Docker image, you can do that too.

Assuming you have Vagrant installed already with a VM that has Docker installed, follow these steps:

vagrant up
vagrant ssh

Install Git to clone the NuPIC repo containing the Dockerfile:

git clone https://github.com/numenta/nupic.git
cd nupic

Build the image:

sudo docker build -t numenta/nupic .

Docker should build the image from the Dockerfile at root. Note: you may need to increase the RAM for your VM in order for this to work. You can interact with your image in this way:

sudo docker run -i -t numenta/nupic /bin/bash

You will now be logged into the freshly built Docker image containing NuPIC and all of its dependencies. You will want to get up to speed on how to work with Docker, so be sure to read the documentation: http://docs.docker.io/en/latest/

Update Your Docker Image

If you want to update your image, you can just rebuild it after pulling the latest NuPIC files. If you think it’s worth the trouble, you can also update it from inside a container and then commit the changes to the image.

Running swarms

See https://github.com/numenta/nupic/wiki/Running-Swarms#running-a-swarm-with-docker for complete instructions for running swarms in Docker.

1 Like

In case this is useful for anyone else - i was getting an error trying to build the dockerfile in the github repo.

It was an error saying

CMake 3.3 or higher is required. You are running version 2.8.12.2

I seem to have resolved this by adding in these lines:

RUN apt remove cmake -y
RUN apt-get install cmake3 -y
RUN cmake --version

Attaching image of the highlighted changed in the dockerfile.

Here is a pull request for the Dockerfile. I actually think you can just use ‘cmake3’ instead of ‘cmake’

1 Like

Sorry pull request is here

1 Like

HI

installing the docker and trying to run the tests, I got the following error:

do you have any idea about this error?

thanks in advance

Arkady

On man I have not been keeping the docker builds up to date. I will have to look into this.

thx a lot!

I have been reading and watched your videos and read some tutorials ( Gym tutorial and HTM seems to be great for our purpose - anomaly detection in IT department in intel ).

I ran our data on the demo program and it worked perfectly ( no false positive ).

meanwhile I will try to install in manually. thx again.

arkady

2 Likes

Use this command. It just needed python added. I updated the wiki above.

docker run numenta/nupic python /usr/local/src/nupic/scripts/run_nupic_tests.py -u --coverage

thx it works!

have a great day

Arkady

‫בתאריך יום ג׳, 5 בפבר׳ 2019 ב-19:22 מאת ‪Matt Taylor via HTM Forum‬‏ <‪numenta@discoursemail.com‬‏>:‬

1 Like

Hi Again,

“it will fail the Python swarming tests, unless you install MySQL in your container” - so i just learned its true:). the issue is from some reason I’m not able to install mysql on the container. Im using - apt-get install mysql-server but I get : Temporary failure resolving ‘archive.ubuntu.com’. it may be related to some network issues here. I wonder if you managed to install it somehow? using this command or other way? and also should I configure the mysql in some particular way? create any database? or the program is just looking for running mysql service?

thanks:)

Arkady

Hi,

any update on this one?
someone got this issue using a docker?
all I have to do is install MySQL in it?

The failure you showed in your previous post did seem like a network error. Did you try again recently? You don’t really need MySQL to run NuPIC unless you plan on swarming, so you can safely ignore this requirement otherwise.

Hi everybody,

I am trying to set up Nupic in Docker. I followed the instructions at the beginning this thread and things seem to work. I can run all the commands, check out the tutorials etc.

More specifically, on running:

sudo docker run -i -t numenta/nupic /bin/bash

it pulls the image from Dockerhub, starts it up, and drops me into the shell. From there, I can open up python interpreter, or create a script or whatever in the container and start playing with Nupic libraries - all good so far.

However I wanted to change some things in the Dockerfile to customise it a little, mostly just to add some applications and libraries I like having in my dev environment - I also wanted to have the container be able to see my local machine files, so set up a bind mount kind of thing, which usually seems really easy with Docker.

But the problem is that if try and build this image directly from the Dockerfile (at: https://hub.docker.com/r/numenta/nupic/dockerfile) rather than pulling it, it doesn’t seem work - when I downloaded the Dockerfile and ran docker image build -t mycustomnupicimage I get the following error:

Collecting setuptools

/tmp/tmpze03H0/pip.zip/pip/vendor/requests/packages/urllib3/util/ssl.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.

Downloading https://files.pythonhosted.org/packages/42/3e/2464120172859e5d103e5500315fb5555b1e908c0dacc73d80d35a9480ca/setuptools-45.1.0.zip (859kB)

Could not import setuptools which is required to install from a source distribution.

Please install setuptools.


Investigating this leads to lots of issues that mostly come python from 2.7 deprecation issues: DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip...

I am trying to get around it by just stating with a python:2.7 image but running into lots of issues.

Just wondering if anyone has some ideas around this - what I really want is to set things up more along the lines of the solution being explored at: Docker, jupyter and HTM , though I can’t make that work either.

Does anyone have any suggestions? - Apologies if I am missing some kind of Docker issue here, its usually pretty plug and play, so I haven’t had to troubleshoot things like this too much.

hang on - I have discovered htm.core which seems to fix all of this - got that installed and working now

2 Likes