NuPIC 0.7.0 macOS 10.10.5 installation problems


#21

Hi @jacobeverist

I installed the MacPorts capnproto and pip pycapnp into Anaconda for pyhton2.7 on my MBPr (mid 2012) with Sierra 10.12.5 without any hassle


#22

I’m having the same issue—been battling with it for the last 2.5 hours. If it’s possible to avoid this dependency, I’d highly recommend doing so…

btw: I’m on macOS 10.12.6, and I’ve tried with both my condo python 2.7 install and the python.framework install


#23

I think we need to provide some instructions on installing NuPIC without the capnp dependency. Do we have any instructions for that @scott or @vkruglikov?


#24

The capnp library is compiled into nupic.bindings and I don’t think that is the problem. It looks like the pycapnp install is failing when it tries to compile its version of capnp. If you can get pycapnp installed then you should be all set. You can also install nupic.bindings with “–no-deps” and simply never install pycapnp and the code will all still work (albeit without serialization support).


#25

Okay, if I can’t get pycapnp installed, I’ll try with --no-deps. I will obviously need serialization if I decide to go the HTM route for my application, but I can certainly try out some ideas without it.

Thanks.


#26

You can fall back to the old serialization options if you want:
http://nupic.docs.numenta.org/stable/guides/serialization.html#deprecated-serialization-methods


#27

Hmm… yeah, this is the error when trying to install capnproto from git (after: make -j6 check):

echo capnp capnpc-c++ src/capnp/test.capnp src/capnp/test-import.capnp src/capnp/test-import2.capnp | (read CAPNP CAPNPC_CXX SOURCES && ./$CAPNP compile --src-prefix=./src -o./$CAPNPC_CXX:src -I./src $SOURCES)
dyld: lazy symbol binding failed: Symbol not found: __ZNKSt8__detail20_Prime_rehash_policy11_M_next_bktEm
  Referenced from: /Users/jbmaxwell/capnproto/c++/.libs/libcapnpc-0.7-dev.dylib (which was built for Mac OS X 10.12)
    Expected in: /usr/lib/libstdc++.6.dylib

dyld: Symbol not found: __ZNKSt8__detail20_Prime_rehash_policy11_M_next_bktEm
  Referenced from: /Users/jbmaxwell/capnproto/c++/.libs/libcapnpc-0.7-dev.dylib (which was built for Mac OS X 10.12)
  Expected in: /usr/lib/libstdc++.6.dylib

*** Received signal #6: Abort trap: 6
stack: 
make: *** [test_capnpc_middleman] Error 1

#28

Sadly, I was not able to find a consistent solution to this problem nor was I able to pin down exactly what caused it. It’s the pycapnp installation that fails because it has c++ compiling in the installation script and has external library dependencies that don’t always work due to header or link issues. It also may be an issue of standard c/c++ libraries and compilers.

I have one OSX machine where it works without issue and I have another OSX machine that fails all the time. I had it work once, but I can’t figure out how to get back to that state with the latest versions. I’ve been forced to deploy NUPIC to Linux VMs and work remotely that way.

I have abandoned trying to work OSX for the time being until this situation has been sorted out. I spent far too much time on it as is :slight_smile:


#29

Thanks for the note. I understand, as I’ve spent a good deal of time already as well. I will probably try to get a Linux distro running. Actually, I have an old(er) PC that’s running as a Hackintosh that I’ve been thinking of upgrading with a new GPU for Keras and (perhaps) NuPIC.


#30

I can also say that I tried to work with the pycapnp developer on this issue but they were completely unresponsive. I think its risky to have NUPIC depend on an unsupported and unmaintained package.
You can find my issue chain on their project page:

https://github.com/capnproto/pycapnp/issues/155


#31

Thank you for filing that bug report. As far as I know, pycapnp is not unmaintained. I will also make a comment on the issue you created noting that multiple people are having similar problems getting going.


#32

That would be great. My keras/theano install was very smooth, and I know you guys want to get users making sexy demos so the ML folks will get more excited about HTM! A pain-free install process would go a long way! :wink:
For myself, I’m coming back to see how HTM is doing after several years away—I was researching HTM (for music) back in the pre-2013 days. The new algorithms look very promising, so I’d definitely like to see what they can do. (And if they work for me, then some sexy demos will certainly be on the way!)


#33

I thought we had one! These pycapnp problems have caused us pain before, but generally pip install nupic has been working for most people. :worried:


#34

Of course, yes. That would clearly be a pretty simple install! Frustrating about this macOS problem… Maybe the capnp people will look into it.


#35

@jbmaxwell @rhyolight

I was able to fix my problem with the recommendation from https://github.com/capnproto/pycapnp/issues/155

Use the following command-line:

CFLAGS="-mmacosx-version-min=10.7 -std=c++11 -stdlib=libc++" pip install pycapnp

This will install the latest version of pycapnp which is 0.6.1 and unfortunately the wrong version for nupic’s dependency.

If I try to install the version of pycapnp that nupic wants:
CFLAGS="-mmacosx-version-min=10.7 -std=c++11 -stdlib=libc++" pip install pycapnp==0.5.12

The install process fails.

I fixed this by downloading the nupic release source and then editing the setup.py to use the 0.6.1 pycapnp instead of the 0.5.12 version. It appears to work correctly.

So the procedure is:

  1. Install pycapnp==0.6.1 separately using the above CFLAGS prefixes.

  2. download nupic release source

  3. edit setup.py to change pycapnp version to 0.6.1

  4. run pip install . in nupic source directory and it will use the installed pycapnp with no problem.


#36

Ah, brilliant! I’ll give this a try, thanks!.

Okay, my problem is that I can’t even install 0.6.1—it always fails with:
/var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/tmp75VP_F/vers.cpp:4:10: fatal error: 'capnp/common.h' file not found #include "capnp/common.h",
then configure: error: in/private/var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/pip-build-4vtZ7a/pycapnp/bundled/capnproto-c++’:
configure: error: C compiler cannot create executables
See config.log' for more details,
and finally:
File "/private/var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/pip-build-4vtZ7a/pycapnp/setup.py", line 124, in run build_libcapnp(bundle_dir, build_dir) File "buildutils/build.py", line 21, in build_libcapnp raise RuntimeError('Configure failed') RuntimeError: Configure failed
??


#37

Show your complete command-line and results please.


#38

…of course, sorry about that (I added --no-cache-dir to avoid any cached version):

Jamess-MacBook-Pro:~ jbmaxwell$ CFLAGS="-mmacosx-version-min=10.7 -std=c++11 -stdlib=libc++" pip install pycapnp --no-cache-dir
Collecting pycapnp
  Downloading pycapnp-0.6.1.tar.gz (454kB)
    100% |████████████████████████████████| 460kB 406kB/s 
Installing collected packages: pycapnp
  Running setup.py install for pycapnp ... error
    Complete output from command /Users/jbmaxwell/anaconda/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/pip-build-4vtZ7a/pycapnp/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/pip-Aajc_I-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.7-x86_64-2.7
    creating build/lib.macosx-10.7-x86_64-2.7/capnp
    copying capnp/__init__.py -> build/lib.macosx-10.7-x86_64-2.7/capnp
    copying capnp/_gen.py -> build/lib.macosx-10.7-x86_64-2.7/capnp
    copying capnp/version.py -> build/lib.macosx-10.7-x86_64-2.7/capnp
    copying capnp/__init__.pxd -> build/lib.macosx-10.7-x86_64-2.7/capnp
    copying capnp/c++.capnp -> build/lib.macosx-10.7-x86_64-2.7/capnp
    copying capnp/schema.capnp -> build/lib.macosx-10.7-x86_64-2.7/capnp
    creating build/lib.macosx-10.7-x86_64-2.7/capnp/helpers
    copying capnp/helpers/__init__.pxd -> build/lib.macosx-10.7-x86_64-2.7/capnp/helpers
    copying capnp/helpers/helpers.pxd -> build/lib.macosx-10.7-x86_64-2.7/capnp/helpers
    copying capnp/helpers/non_circular.pxd -> build/lib.macosx-10.7-x86_64-2.7/capnp/helpers
    copying capnp/helpers/asyncHelper.h -> build/lib.macosx-10.7-x86_64-2.7/capnp/helpers
    copying capnp/helpers/capabilityHelper.h -> build/lib.macosx-10.7-x86_64-2.7/capnp/helpers
    copying capnp/helpers/checkCompiler.h -> build/lib.macosx-10.7-x86_64-2.7/capnp/helpers
    copying capnp/helpers/fixMaybe.h -> build/lib.macosx-10.7-x86_64-2.7/capnp/helpers
    copying capnp/helpers/rpcHelper.h -> build/lib.macosx-10.7-x86_64-2.7/capnp/helpers
    copying capnp/helpers/serialize.h -> build/lib.macosx-10.7-x86_64-2.7/capnp/helpers
    creating build/lib.macosx-10.7-x86_64-2.7/capnp/includes
    copying capnp/includes/__init__.pxd -> build/lib.macosx-10.7-x86_64-2.7/capnp/includes
    copying capnp/includes/capnp_cpp.pxd -> build/lib.macosx-10.7-x86_64-2.7/capnp/includes
    copying capnp/includes/schema_cpp.pxd -> build/lib.macosx-10.7-x86_64-2.7/capnp/includes
    copying capnp/includes/types.pxd -> build/lib.macosx-10.7-x86_64-2.7/capnp/includes
    creating build/lib.macosx-10.7-x86_64-2.7/capnp/lib
    copying capnp/lib/__init__.pxd -> build/lib.macosx-10.7-x86_64-2.7/capnp/lib
    copying capnp/lib/capnp.pxd -> build/lib.macosx-10.7-x86_64-2.7/capnp/lib
    copying capnp/lib/__init__.py -> build/lib.macosx-10.7-x86_64-2.7/capnp/lib
    copying capnp/lib/pickle_helper.py -> build/lib.macosx-10.7-x86_64-2.7/capnp/lib
    copying capnp/lib/capnp.pyx -> build/lib.macosx-10.7-x86_64-2.7/capnp/lib
    creating build/lib.macosx-10.7-x86_64-2.7/capnp/templates
    copying capnp/templates/module.pyx -> build/lib.macosx-10.7-x86_64-2.7/capnp/templates
    copying capnp/templates/setup.py.tmpl -> build/lib.macosx-10.7-x86_64-2.7/capnp/templates
    running build_ext
    creating var
    creating var/folders
    creating var/folders/pj
    creating var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn
    creating var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T
    creating var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/tmp75VP_F
    gcc -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -mmacosx-version-min=10.7 -std=c++11 -stdlib=libc++ -c /var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/tmp75VP_F/vers.cpp -o var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/tmp75VP_F/vers.o --std=c++11
    /var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/tmp75VP_F/vers.cpp:4:10: fatal error: 'capnp/common.h' file not found
    #include "capnp/common.h"
             ^~~~~~~~~~~~~~~~
    1 error generated.
    *WARNING* no libcapnp detected or rebuild forced. Will download and build it from source now. If you have C++ Cap'n Proto installed, it may be out of date or is not being detected. Downloading and building libcapnp may take a while.
    fetching https://capnproto.org/capnproto-c++-0.6.0.tar.gz into /private/var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/pip-build-4vtZ7a/pycapnp/bundled
    configure: error: in `/private/var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/pip-build-4vtZ7a/pycapnp/bundled/capnproto-c++':
    configure: error: C compiler cannot create executables
    See `config.log' for more details
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/pip-build-4vtZ7a/pycapnp/setup.py", line 180, in <module>
        'Topic :: Communications'],
      File "/Users/jbmaxwell/anaconda/lib/python2.7/distutils/core.py", line 151, in setup
        dist.run_commands()
      File "/Users/jbmaxwell/anaconda/lib/python2.7/distutils/dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "/Users/jbmaxwell/anaconda/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/Users/jbmaxwell/anaconda/lib/python2.7/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/Users/jbmaxwell/anaconda/lib/python2.7/distutils/command/install.py", line 563, in run
        self.run_command('build')
      File "/Users/jbmaxwell/anaconda/lib/python2.7/distutils/cmd.py", line 326, in run_command
        self.distribution.run_command(command)
      File "/Users/jbmaxwell/anaconda/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/Users/jbmaxwell/anaconda/lib/python2.7/distutils/command/build.py", line 127, in run
        self.run_command(cmd_name)
      File "/Users/jbmaxwell/anaconda/lib/python2.7/distutils/cmd.py", line 326, in run_command
        self.distribution.run_command(command)
      File "/Users/jbmaxwell/anaconda/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/private/var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/pip-build-4vtZ7a/pycapnp/setup.py", line 124, in run
        build_libcapnp(bundle_dir, build_dir)
      File "buildutils/build.py", line 21, in build_libcapnp
        raise RuntimeError('Configure failed')
    RuntimeError: Configure failed
    
    ----------------------------------------
Command "/Users/jbmaxwell/anaconda/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/pip-build-4vtZ7a/pycapnp/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/pip-Aajc_I-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/pj/2swl58m14dldg6mmc4f9g4qc0000gn/T/pip-build-4vtZ7a/pycapnp/`

#39

Have you satisfied the dependencies for OSX? Particularly, the a clang compiler version and xcode tools?

See the README at the bottom: https://github.com/numenta/nupic

Dependencies

The following dependencies are required to install NuPIC on all operating systems.

Python 2.7
pip>=8.1.2
setuptools>=25.2.0
wheel>=0.29.0
numpy

C++ 11 compiler like gcc (4.8+) or clang
Additional OS X requirements:

Xcode command line tools

#40

I’ll double-check, but I’d be very surprised if anything was missing…