Compiling NuPIC on Ubuntu 14

compilation
ubuntu

#1

Update apt-get

sudo apt-get update
sudo apt-get upgrade # optional

Setup Python 2.7 Development Environment

This command requires administrator access.

sudo apt-get install build-essential python-dev libsqlite3-dev libreadline6-dev libgdbm-dev zlib1g-dev libbz2-dev sqlite3 zip

Install required packages

These may already be installed, depending on your system. This command requires administrator access.

sudo apt-get install python-pip mysql-server

Now that pip is installed, install numpy.

pip install numpy

NOTE: During this installation, you may be prompted to enter a MySQL password for the root user. If you provide your own root password, you will need to update your MySQL settings in the NuPIC configuration file.

  • git: for cloning the NuPIC repository
  • python-dev: to get Python headers for compiling C++ with SWIG
  • python-pip: for python package installation of required NuPIC python dependencies
  • automake: for C++ build
  • libtool: GNU tool for creating portable compiled binaries
  • libssl-dev: encryption libraries
  • g++: GNU C++ compiler
  • cmake: cross-platform, open-source make system
  • mysql-server: Used for swarming
  • libpcre3-dev: Perl Compatible Regular Expressions development library (for SWIG compilation below, only required for 32b systems)
  • numpy: Python math package

NOTE: You may also need to install numpy to get your installation to work. Personally, I used sudo apt-get install numpy since I didn’t know exactly which package caused the install to complain.

Install GCC 4.8

gcc4.8 is required for C++11. You may do this however you like. But here are some instructions for one way of doing it.

** gcc-4.8 is required by NuPIC now ** and it is also default on most Ubuntu systems (14.04 LTS +)

  1. sudo add-apt-repository ppa:ubuntu-toolchain-r/test
  2. sudo apt-get update
  3. sudo apt-get install gcc-4.8 g++-4.8
  4. sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.6 60 --slave /usr/bin/g++ g++ /usr/bin/g++-4.6
  5. sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 40 --slave /usr/bin/g++ g++ /usr/bin/g++-4.8
  6. sudo update-alternatives --config gcc (Choose the option for the new 4.8 compiler)

Clone the NuPIC repo

Change to the directory where you want to keep the nupic repo:

git clone https://github.com/numenta/nupic.git

This will create a nupic directory containing the NuPIC source code in the current directory.

Set the NUPIC environment variable

export NUPIC=$PWD/nupic

Build NuPIC

python setup.py install --user

Troubleshooting

If you have problems with the python setup.py install command, try cleaning out your local checkout by running git clean -dfx. WARNING: This will clear out any local files that are not controlled by git.

Not enough memory for compilation

If you run in VM (or on a system with quite low RAM), for compilation it seems the gcc requires atleast 2GB RAM memory, otherwise you can see errors like in this thread


NuPIC Wiki is being deprecated
When building NUPIC getting error undefined references to YAML in 64-bit Ubuntu
#2

@matt, this text that looks like a URL doesn’t seem to go anywhere.


#3

Thanks, fixed.


#4

Quick Install Script

If you don’t care what’s actually happening in the installation described below, you can just run this script (with sudo rights!) on a freshly installed Ubuntu 14 LTS, and it should do everything you need. However, I suggest you install NuPIC by following the step-by-step directions below so you’ll know exactly what the installation process does in your environment.

Setup

Install some basic depedencies

sudo apt-get update -y
sudo apt-get install git g++ cmake python-dev -y

Clone nupic.core and nupic

git clone https://github.com/numenta/nupic.core.git
git clone https://github.com/numenta/nupic.git

Set some environment vars

These point to our checkout directories. We’ll need them later for running tests and scripts. (You’ll probably want this in your .bashrc or .bash_profile.)

export NUPIC=$HOME/nupic
export NUPIC_CORE=$HOME/nupic.core

Get the latest pip

Notice we are not using apt-get because it provides a very old version of pip that is hard to update properly.

curl https://bootstrap.pypa.io/get-pip.py | sudo python

NuPIC Core

Install nupic.core python dependencies

This takes awhile because numpy!

cd $NUPIC_CORE
pip install -r bindings/py/requirements.txt --user

Install pycapnp

This will install the python Capnproto bindings and the C++ Capnproto program.

pip install pycapnp==0.5.8 --user

Configure and generate C++ build files

See the README for details about the following commands.

mkdir -p $NUPIC_CORE/build/scripts
cd $NUPIC_CORE/build/scripts
cmake $NUPIC_CORE -DCMAKE_INSTALL_PREFIX=../release -DPY_EXTENSIONS_DIR=$NUPIC_CORE/bindings/py/nupic/bindings

Build

Note: The -j3 option specifies ‘3’ as the maximum number of parallel jobs/threads that Make will use during the build in order to gain speed. However, you can increase this number depending your CPU.

make -j3

Install

Stay in the $NUPIC_CORE/build/scripts directory.

make install

Run Tests

cd $NUPIC_CORE/build/release/bin
./cpp_region_test
./unit_tests

Install the Python Bindings

This will expose the C++ nupic.core program to python by installing the nupic.bindings python module.

cd $NUPIC_CORE
python setup.py install --user

NuPIC

Install

cd $NUPIC
python setup.py install --user

Run Tests

First, make sure that py.test is on your path. It got installed into ~/.local/bin in a previous step.

export PATH=~/.local/bin:$PATH

Now run unit tests.

python $NUPIC/scripts/run_nupic_tests.py -u

Note:
If executing that last line generates an error such as “ImportError: No module named _markerlib”, it may be caused by Ubuntu’s pre-installed pip version which seems to be outdated. Running these 3 lines should fix the problem.

sudo apt-get install python-pip python-dev build-essential
sudo pip install --upgrade pip
sudo pip install --upgrade virtualenv

Running swarms

To run swarms, a MySQL database is necessary. If MySQL is already installed and has already been setup with a username and password, then you will need to modify the user and password values accordingly in the file nupic-default.xml located at $NUPIC/config/default/nupic-default.xml. However, if it is not installed, then simply executing

sudo apt-get install mysql-server

will install MySQL on Ubuntu. During installation, you will be prompted to enter a password for the “root” user. Simply hit the Enter button to complete installation. By default, this should create a MySQL database located at “localhost” with the username “root” and an empty password. Once done, you can verify that everything was setup correctly by executing

python $NUPIC/examples/swarm/test_db.py

which should generate some text, with the very last line confirming that the connection test was successful.

Note:
To run swarms, the script run_swarm.py must be executed. For this to work, you must either add the script run_swarm.py to the python path or simply invoke the entire path everytime as such

$NUPIC/scripts/run_swarm.py $NUPIC/your_folder/yoursearch_def.json --maxWorkers=4

More details and in-depth explanation about swarms and how to run them can be found here


Run test fail on Ubuntu 14 LTS