Compiling NuPIC on Ubuntu 14

Engineering NuPIC
,
1

Update apt-get

sudo apt-get update
sudo apt-get upgrade # optional

Setup Python 2.7 Development Environment

This command requires administrator access.

sudo apt-get install build-essential python-dev libsqlite3-dev libreadline6-dev libgdbm-dev zlib1g-dev libbz2-dev sqlite3 zip

Install required packages

These may already be installed, depending on your system. This command requires administrator access.

sudo apt-get install python-pip mysql-server

Now that pip is installed, install numpy.

pip install numpy

NOTE: During this installation, you may be prompted to enter a MySQL password for the root user. If you provide your own root password, you will need to update your MySQL settings in the NuPIC configuration file.

  • git: for cloning the NuPIC repository
  • python-dev: to get Python headers for compiling C++ with SWIG
  • python-pip: for python package installation of required NuPIC python dependencies
  • automake: for C++ build
  • libtool: GNU tool for creating portable compiled binaries
  • libssl-dev: encryption libraries
  • g++: GNU C++ compiler
  • cmake: cross-platform, open-source make system
  • mysql-server: Used for swarming
  • libpcre3-dev: Perl Compatible Regular Expressions development library (for SWIG compilation below, only required for 32b systems)
  • numpy: Python math package

NOTE: You may also need to install numpy to get your installation to work. Personally, I used sudo apt-get install numpy since I didn’t know exactly which package caused the install to complain.

Install GCC 4.8

gcc4.8 is required for C++11. You may do this however you like. But here are some instructions for one way of doing it.

** gcc-4.8 is required by NuPIC now ** and it is also default on most Ubuntu systems (14.04 LTS +)

  1. sudo add-apt-repository ppa:ubuntu-toolchain-r/test
  2. sudo apt-get update
  3. sudo apt-get install gcc-4.8 g++-4.8
  4. sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.6 60 --slave /usr/bin/g++ g++ /usr/bin/g++-4.6
  5. sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 40 --slave /usr/bin/g++ g++ /usr/bin/g++-4.8
  6. sudo update-alternatives --config gcc (Choose the option for the new 4.8 compiler)

Clone the NuPIC repo

Change to the directory where you want to keep the nupic repo:

git clone https://github.com/numenta/nupic.git

This will create a nupic directory containing the NuPIC source code in the current directory.

Set the NUPIC environment variable

export NUPIC=$PWD/nupic

Build NuPIC

python setup.py install --user

Troubleshooting

If you have problems with the python setup.py install command, try cleaning out your local checkout by running git clean -dfx. WARNING: This will clear out any local files that are not controlled by git.

Not enough memory for compilation

If you run in VM (or on a system with quite low RAM), for compilation it seems the gcc requires atleast 2GB RAM memory, otherwise you can see errors like in this thread

2

@matt, this text that looks like a URL doesn’t seem to go anywhere.

3

Thanks, fixed.

4

Quick Install Script

If you don’t care what’s actually happening in the installation described below, you can just run this script (with sudo rights!) on a freshly installed Ubuntu 14 LTS, and it should do everything you need. However, I suggest you install NuPIC by following the step-by-step directions below so you’ll know exactly what the installation process does in your environment.

Setup

Install some basic depedencies

sudo apt-get update -y
sudo apt-get install git g++ cmake python-dev -y

Clone nupic.core and nupic

git clone https://github.com/numenta/nupic.core.git
git clone https://github.com/numenta/nupic.git

Set some environment vars

These point to our checkout directories. We’ll need them later for running tests and scripts. (You’ll probably want this in your .bashrc or .bash_profile.)

export NUPIC=$HOME/nupic
export NUPIC_CORE=$HOME/nupic.core

Get the latest pip

Notice we are not using apt-get because it provides a very old version of pip that is hard to update properly.

curl https://bootstrap.pypa.io/get-pip.py | sudo python

NuPIC Core

Install nupic.core python dependencies

This takes awhile because numpy!

cd $NUPIC_CORE
pip install -r bindings/py/requirements.txt --user

Install pycapnp

This will install the python Capnproto bindings and the C++ Capnproto program.

pip install pycapnp==0.5.8 --user

Configure and generate C++ build files

See the README for details about the following commands.

mkdir -p $NUPIC_CORE/build/scripts
cd $NUPIC_CORE/build/scripts
cmake $NUPIC_CORE -DCMAKE_INSTALL_PREFIX=../release -DPY_EXTENSIONS_DIR=$NUPIC_CORE/bindings/py/nupic/bindings

Build

Note: The -j3 option specifies ‘3’ as the maximum number of parallel jobs/threads that Make will use during the build in order to gain speed. However, you can increase this number depending your CPU.

make -j3

Install

Stay in the $NUPIC_CORE/build/scripts directory.

make install

Run Tests

cd $NUPIC_CORE/build/release/bin
./cpp_region_test
./unit_tests

Install the Python Bindings

This will expose the C++ nupic.core program to python by installing the nupic.bindings python module.

cd $NUPIC_CORE
python setup.py install --user

NuPIC

Install

cd $NUPIC
python setup.py install --user

Run Tests

First, make sure that py.test is on your path. It got installed into ~/.local/bin in a previous step.

export PATH=~/.local/bin:$PATH

Now run unit tests.

python $NUPIC/scripts/run_nupic_tests.py -u

Note:
If executing that last line generates an error such as “ImportError: No module named _markerlib”, it may be caused by Ubuntu’s pre-installed pip version which seems to be outdated. Running these 3 lines should fix the problem.

sudo apt-get install python-pip python-dev build-essential
sudo pip install --upgrade pip
sudo pip install --upgrade virtualenv

Running swarms

To run swarms, a MySQL database is necessary. If MySQL is already installed and has already been setup with a username and password, then you will need to modify the user and password values accordingly in the file nupic-default.xml located at $NUPIC/config/default/nupic-default.xml. However, if it is not installed, then simply executing

sudo apt-get install mysql-server

will install MySQL on Ubuntu. During installation, you will be prompted to enter a password for the “root” user. Simply hit the Enter button to complete installation. By default, this should create a MySQL database located at “localhost” with the username “root” and an empty password. Once done, you can verify that everything was setup correctly by executing

python $NUPIC/examples/swarm/test_db.py

which should generate some text, with the very last line confirming that the connection test was successful.

Note:
To run swarms, the script run_swarm.py must be executed. For this to work, you must either add the script run_swarm.py to the python path or simply invoke the entire path everytime as such

$NUPIC/scripts/run_swarm.py $NUPIC/your_folder/yoursearch_def.json --maxWorkers=4

More details and in-depth explanation about swarms and how to run them can be found here