I have tried installing Nupic on multiple operating systems and I can’t get this demo to work… Preferably I’d want it to work on my Ubuntu15.10.
When I run swarm.py I get an error saying that python can’t find the nupic.swarming module. I think the main problem I have is that the bindings aren’t where they are supposed to be and/or they’re not working because of that. I tried reinstalling stuff and going through the tutorial on Ubuntu and the other tutorial again, but I still get it to work.
I can’t help you solve this problem, but I was wondering have you tried the Docker file? It’s a self-contained OS (Linux with NuPIC installed, I believe) that is guaranteed to work (afaik)? Also, you can always run HTM.java’s Examples which are the same thing only in Java; and require no installation?
I don’t think you have NuPIC installed properly. How did you go about installing it?
To install Nupic,I followed thse tutorials:
and
In the video, when you do the installation of the wheel file, you then run the pip install nupic --user command, but unlike you, I did have an error. It was complaining about bindings.
However, if I do pip freeze | grep python then I get nupic==0.5.3.dev0; nupic.bindings==0.4.3.
So we need to start there.
First, uninstall everything by running these two commands a few times just to make sure they are uninstalled:
pip uninstall nupic -y
pip uninstall nupic.bindings -y
Then run this to install the nupic.core binary bindings (called nupic.bindings) from S3:
pip install https://s3-us-west-2.amazonaws.com/artifacts.numenta.org/numenta/nupic.core/releases/nupic.bindings/nupic.bindings-0.4.3-cp27-none-linux_x86_64.whl
If you truly have a problem installing nupic.bindings, that command should fail in some way. Please paste the error here so we can discuss.
Also, can you tell me exactly what version of Ubuntu you are running?
I am using Kubuntu15.10, Linux 4.2.0-36-generic #42-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux
I did get an error, here’s what happened when I ran the last command:
$ pip install https://s3-us-west-2.amazonaws.com/artifacts.numenta.org/numenta/nupic.core/releases/nupic.bindings/nupic.bindings-0.4.3-cp27-none-linux_x86_64.whl
Collecting nupic.bindings==0.4.3 from https://s3-us-west-2.amazonaws.com/artifacts.numenta.org/numenta/nupic.core/releases/nupic.bindings/nupic.bindings-0.4.3-cp27-none-linux_x86_64.whl
Using cached https://s3-us-west-2.amazonaws.com/artifacts.numenta.org/numenta/nupic.core/releases/nupic.bindings/nupic.bindings-0.4.3-cp27-none-linux_x86_64.whl
Requirement already satisfied (use --upgrade to upgrade): pytest-cov==1.6 in /home/misa/.local/lib/python2.7/site-packages (from nupic.bindings==0.4.3)
Collecting numpy==1.9.2 (from nupic.bindings==0.4.3)
Using cached numpy-1.9.2-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied (use --upgrade to upgrade): pytest-xdist==1.8 in /home/misa/.local/lib/python2.7/site-packages (from nupic.bindings==0.4.3)
Collecting pytest==2.5.1 (from nupic.bindings==0.4.3)
Requirement already satisfied (use --upgrade to upgrade): cov-core>=1.6 in /home/misa/.local/lib/python2.7/site-packages (from pytest-cov==1.6->nupic.bindings==0.4.3)
Requirement already satisfied (use --upgrade to upgrade): execnet>=1.0.8 in /home/misa/.local/lib/python2.7/site-packages (from pytest-xdist==1.8->nupic.bindings==0.4.3)
Requirement already satisfied (use --upgrade to upgrade): py>=1.4.19 in /usr/lib/python2.7/dist-packages (from pytest==2.5.1->nupic.bindings==0.4.3)
Requirement already satisfied (use --upgrade to upgrade): coverage>=3.6 in /home/misa/.local/lib/python2.7/site-packages/coverage-3.7.1-py2.7-linux-x86_64.egg (from cov-core>=1.6->pytest-cov==1.6->nupic.bindings==0.4.3)
Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /home/misa/.local/lib/python2.7/site-packages (from execnet>=1.0.8->pytest-xdist==1.8->nupic.bindings==0.4.3)
Installing collected packages: numpy, pytest, nupic.bindings
Found existing installation: numpy 1.8.2
DEPRECATION: Uninstalling a distutils installed project (numpy) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
Uninstalling numpy-1.8.2:
Exception:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 736, in install
requirement.uninstall(auto_confirm=True)
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 742, in uninstall
paths_to_remove.remove(auto_confirm)
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/usr/local/lib/python2.7/dist-packages/pip/utils/__init__.py", line 267, in renames
shutil.move(old, new)
File "/usr/lib/python2.7/shutil.py", line 303, in move
os.unlink(src)
OSError: [Errno 13] Permission denied: '/usr/lib/python2.7/dist-packages/numpy-1.8.2.egg-info'
That error is because you don’t have permission to install python modules with pip into a system location. There are two things to try.
First thing to try is running that same command, but with --user at the end of it, which will attempt to install into a non-system location.
If you still get an error, the brute force way to do it is to use sudo with the pip command (without the --user).
Let me know if either of those tactics help out.
Okay, I installed it with the “–user”
I still get the same error with swarm.py though
An no error?
Right, because we are not done yet. 
Now you run pip install nupic --user. If that works, set the $NUPIC env var to your checkout directory and run python $NUPIC/examples/swarm/test_db.py to see if you have MySQL set up right.
“Successfully installed numpy-1.8.2 nupic.bindings-0.4.3 pytest-2.7.2”
“Connection successful”
Now I get the same error I get on my Windows machine when I run swarm.py:
$ python swarm.py
This script runs a swarm on the input data (rec-center-hourly.csv) and
creates a model parameters file in the `model_params` directory containing
the best model found by the swarm. Dumps a bunch of crud to stdout because
that is just what swarming does at this point. You really don't need to
pay any attention to it.
=================================================
= Swarming on rec-center-hourly data...
= Medium swarm. Sit back and relax, this could take awhile.
=================================================
Generating experiment files in directory: /home/misa/Desktop/My Stuff/programming stuff/Nupic/nupic-master/examples/opf/clients/hotgym/prediction/one_gym/swarm...
Writing 313 lines...
Writing 114 lines...
done.
None
Successfully submitted new HyperSearch job, jobID=1010
Evaluated 0 models
HyperSearch finished!
Worker completion message: None
Results from all experiments:
----------------------------------------------------------------
Generating experiment files in directory: /tmp/tmp0lzaX6...
Writing 313 lines...
Writing 114 lines...
done.
None
json.loads(jobInfo.results) raised an exception. Here is some info to help with debugging:
jobInfo: _jobInfoNamedTuple(jobId=1010, client=u'GRP', clientInfo=u'', clientKey=u'', cmdLine=u'$HYPERSEARCH', params=u'{"hsVersion": "v2", "maxModels": null, "persistentJobGUID": "c1cfb836-1faf-11e6-812a-ac7ba1e42884", "useTerminators": false, "description": {"includedFields": [{"fieldName": "timestamp", "fieldType": "datetime"}, {"maxValue": 53.0, "fieldName": "kw_energy_consumption", "fieldType": "float", "minValue": 0.0}], "streamDef": {"info": "kw_energy_consumption", "version": 1, "streams": [{"info": "Rec Center", "source": "file://rec-center-hourly.csv", "columns": ["*"]}]}, "inferenceType": "TemporalMultiStep", "inferenceArgs": {"predictionSteps": [1], "predictedField": "kw_energy_consumption"}, "iterationCount": -1, "swarmSize": "medium"}}', jobHash='\xc1\xcf\xb87\x1f\xaf\x11\xe6\x81*\xac{\xa1\xe4(\x84', status=u'notStarted', completionReason=None, completionMsg=None, workerCompletionReason=u'success', workerCompletionMsg=None, cancel=0, startTime=None, endTime=None, results=None, engJobType=u'hypersearch', minimumWorkers=1, maximumWorkers=4, priority=0, engAllocateNewWorkers=1, engUntendedDeadWorkers=0, numFailedWorkers=0, lastFailedWorkerErrorMsg=None, engCleaningStatus=u'notdone', genBaseDescription=None, genPermutations=None, engLastUpdateTime=datetime.datetime(2016, 5, 21, 23, 57, 21), engCjmConnId=None, engWorkerState=None, engStatus=None, engModelMilestones=None)
jobInfo.results: None
EXCEPTION: expected string or buffer
Traceback (most recent call last):
File "swarm.py", line 109, in <module>
swarm(INPUT_FILE)
File "swarm.py", line 101, in swarm
modelParams = swarmForBestModelParams(SWARM_DESCRIPTION, name)
File "swarm.py", line 78, in swarmForBestModelParams
verbosity=0
File "/home/misa/.local/lib/python2.7/site-packages/nupic/swarming/permutations_runner.py", line 277, in runWithConfig
return _runAction(runOptions)
File "/home/misa/.local/lib/python2.7/site-packages/nupic/swarming/permutations_runner.py", line 218, in _runAction
returnValue = _runHyperSearch(runOptions)
File "/home/misa/.local/lib/python2.7/site-packages/nupic/swarming/permutations_runner.py", line 161, in _runHyperSearch
metricsKeys=search.getDiscoveredMetricsKeys())
File "/home/misa/.local/lib/python2.7/site-packages/nupic/swarming/permutations_runner.py", line 828, in generateReport
results = json.loads(jobInfo.results)
File "/home/misa/.local/lib/python2.7/site-packages/nupic/swarming/object_json.py", line 163, in loads
json.loads(s, object_hook=objectDecoderHook, **kwargs))
File "/usr/lib/python2.7/json/__init__.py", line 351, in loads
return cls(encoding=encoding, **kw).decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer
I hate that error. I’ve seen it before:
Can you try one thing… set an environment variable called NUPIC to point to the absolute path to your nupic code checkout and try again. If that doesn’t work, one more thing to try in this comment:
It doesn’t work with the absolute path, i.e.,:
$ echo $NUPIC
/home/misa/Desktop/My-Stuff/programming-stuff/Nupic/nupic-master
and it doesn’t work with:
$ echo $NUPIC
/home/misa/Desktop/My-Stuff/programming-stuff/Nupic/nupic-master/examples/opf/clients/hotgym/prediction/one_gym
It didn’t work, with this environment variable either:
$ echo $NUPIC
/home/misa/.local/lib/python2.7/site-packages/nupic
So where we just supposed to run that code as Pehlert said? Or where we supposed to replace “Location” with something else? I just ran it as he said we should run it.
I also tried running it with “sudo -s” enabled as suggested by Lovekeshvig but that didn’t work either.
This doesn’t look good. I haven’t tried getting NuPIC installed on Kubuntu in a long time. Can you try running the simple hotgym example without swarming?
> cd examples/opf/clients/hotgym/simple
> python hotgym.py
INFO:__main__:After 100 records, 1-step altMAPE=38.390240
INFO:__main__:After 200 records, 1-step altMAPE=29.214768
INFO:__main__:After 300 records, 1-step altMAPE=25.829702
INFO:__main__:After 400 records, 1-step altMAPE=24.262638
INFO:__main__:After 500 records, 1-step altMAPE=22.976900
INFO:__main__:After 600 records, 1-step altMAPE=22.890111
INFO:__main__:After 700 records, 1-step altMAPE=22.579927
INFO:__main__:After 800 records, 1-step altMAPE=22.859475
INFO:__main__:After 900 records, 1-step altMAPE=23.299378
INFO:__main__:After 1000 records, 1-step altMAPE=23.396538
If you get output like this, NuPIC is probably installed properly. There may still be a problem with swarming, however.
$ python hotgym.py
Traceback (most recent call last):
File "hotgym.py", line 97, in <module>
runHotgym()
File "hotgym.py", line 82, in runHotgym
result = model.run(modelInput)
File "/usr/lib/python2.7/dist-packages/nupic/frameworks/opf/clamodel.py", line 384, in run
self._sensorCompute(inputRecord)
File "/usr/lib/python2.7/dist-packages/nupic/frameworks/opf/clamodel.py", line 469, in _sensorCompute
sensor.compute()
File "/usr/lib/python2.7/dist-packages/nupic/engine/__init__.py", line 446, in compute
return self._region.compute()
File "/home/misa/.local/lib/python2.7/site-packages/nupic/bindings/engine_internal.py", line 1393, in compute
return _engine_internal.Region_compute(self)
AttributeError: 'RecordSensor' object has no attribute 'guardedCompute'
I forgot to tell you to re-run pip install nupic --user again!
Installing collected packages: numpy, pytest
Successfully installed numpy-1.8.2 pytest-2.7.2
I still get the same error as listed above.
I can get onto an IRC if you want.
Sorry, but I can’t right now. I have never seen that error before. I’m not sure what it means, but I have a suspicion that there is something wrong with the nupic.bindings installation. Let’s try from scratch one more time in case we polluted something along the way…
# run these twice just to ensure it's really gone
> pip uninstall nupic -y
> pip uninstall nupic -y
> pip uninstall nupic.bindings -y
> pip uninstall nupic.bindings -y
> pip install https://s3-us-west-2.amazonaws.com/artifacts.numenta.org/numenta/nupic.core/releases/nupic.bindings/nupic.bindings-0.4.3-cp27-none-linux_x86_64.whl --user
> pip install nupic --user
Please paste the complete output of all these comments into gist or pastebin.
Now that I think about it… the issue is probably because the binary was built on Ubuntu and you are trying to run it on Kubuntu. I don’t know about the architectural differences between these two platforms, but if that is the case, you’re going to need to compile from scratch on your Kubuntu system. This tutorial will help you identify the dependencies you’ll need to have installed to compile, but it won’t provide verbatim instructions.
I get an error when doing the tutorial, at the “make -j4” step. The error I get is posted on gist.github. Then of course if I run the command after that “make install” then I also get an error (second thing on gist.github). So I can’t run the command after that “cd $NUPIC_CORE/build/release/bin” because the “release” directory doesn’t exist in the “build” directory.