Building error nupic.core in Ubuntu 16.04 LTS

after upgrading from Ubuntu 14.04 to 16.04 LTS, I compiled the nupic.core again and have got the following error with Apr1StaticLib:

[  4%] Performing configure step for 'CapnProto'
patching file CMakeLists.txt
Hunk #1 FAILED at 248.
1 out of 1 hunk FAILED -- saving rejects to file CMakeLists.txt.rej
patching file include/apr_version.h
Hunk #1 FAILED at 107.
1 out of 1 hunk FAILED -- saving rejects to file include/apr_version.h.rej
external/CMakeFiles/Apr1StaticLib.dir/build.make:119: recipe for target 'ThirdParty/Stamp/Apr1StaticLib/Apr1StaticLib-patch_sources' failed
make[2]: *** [ThirdParty/Stamp/Apr1StaticLib/Apr1StaticLib-patch_sources] Error 1
CMakeFiles/Makefile2:171: recipe for target 'external/CMakeFiles/Apr1StaticLib.dir/all' failed
make[1]: *** [external/CMakeFiles/Apr1StaticLib.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
-- The CXX compiler identification is GNU 5.4.0
-- Check for working CXX compiler: /usr/bin/c++

Does anyone have an idea for solving this problem?
Best thanks, Binh

You might need to manually uninstall / install Capnproto after this upgrade. The old version of Capnp binaries you had previously installed might not work with the latest Ubuntu.

If @rhyolight’s suggestion doesn’t help, then do a clean re-configuration and build of nupic.core (remove nupic.core/build/ directory and start from the beginning). Run a single-process make (make -j1) for more straightforward make processing and easier-to-examine make output. Then post a link to the gist containing the entire cmake and make outputs. Also, post the version (or SHA) of the nupic.core sources that you used for building.

1 Like

how can I uninstall Capnproto manually?
thanks Binh

Let me google that for you :stuck_out_tongue: https://capnproto.org/install.html

I did as you suggestion by removing build directory, an install from the beginning. As I informed last mail, the compiling of Apr1StaticLib is failed with the following error:
emulation options:
No emulation specific options
/usr/bin/ar: supported targets: elf64-x86-64 elf32-i386 elf32-iamcu elf32-x86-64 a.out-i386-linux pei-i386 pei-x86-64 elf64-l1om elf64-k1om elf64-little elf64-big elf32-little elf32-big pe-x86-64 pe-bigobj-x86-64 pe-i386 plugin srec symbolsrec verilog tekhex binary ihex
Makefile:102: recipe for target ‘libapr-1.la’ failed
make[4]: *** [libapr-1.la] Error 1
/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Build/Apr1StaticLib/build/apr_rules.mk:118: recipe for target ‘all-recursive’ failed
make[3]: *** [all-recursive] Error 1
external/CMakeFiles/Apr1StaticLib.dir/build.make:114: recipe for target ‘ThirdParty/Stamp/Apr1StaticLib/Apr1StaticLib-build’ failed
make[2]: *** [ThirdParty/Stamp/Apr1StaticLib/Apr1StaticLib-build] Error 2
CMakeFiles/Makefile2:171: recipe for target ‘external/CMakeFiles/Apr1StaticLib.dir/all’ failed
make[1]: *** [external/CMakeFiles/Apr1StaticLib.dir/all] Error 2

I remember this error looks like what I had by building nupic.core under windows 64bits.
Any idea?
thanks, Binh

I saw that the building Apr1StaticLib uses
/usr/bin/ar with option ‘R’, which is invalid by the current version of GNU ar (GNU Binutils for Ubuntu) 2.26.1, used in Ubuntu 16.04.

Do you know what does this option in the old version mean?
Maybe we have to modify the Cmakefiles using the new option.

Where is the gist with the complete output from cmake and make?

here is:

binh@volkswagen:~/nupic/nupic.core/build/scripts$ make -j1
[  3%] Built target CapnProto
[  6%] Built target ZStaticLib
[  7%] Performing build step for 'Apr1StaticLib'
/usr/bin/ar: invalid option -- 'R'
Usage: /usr/bin/ar [emulation options] [-]{dmpqrstx}[abcDfilMNoPsSTuvV] [--plugin <name>] [member-name] [count] archive-file file...
       /usr/bin/ar -M [<mri-script]
 commands:
  d            - delete file(s) from the archive
  m[ab]        - move file(s) in the archive
  p            - print file(s) found in the archive
  q[f]         - quick append file(s) to the archive
  r[ab][f][u]  - replace existing or insert new file(s) into the archive
  s            - act as ranlib
  t            - display contents of archive
  x[o]         - extract file(s) from the archive
 command specific modifiers:
  [a]          - put file(s) after [member-name]
  [b]          - put file(s) before [member-name] (same as [i])
  [D]          - use zero for timestamps and uids/gids (default)
  [U]          - use actual timestamps and uids/gids
  [N]          - use instance [count] of name
  [f]          - truncate inserted file names
  [P]          - use full path names when matching
  [o]          - preserve original dates
  [u]          - only replace files that are newer than current archive contents
 generic modifiers:
  [c]          - do not warn if the library had to be created
  [s]          - create an archive index (cf. ranlib)
  [S]          - do not build a symbol table
  [T]          - make a thin archive
  [v]          - be verbose
  [V]          - display the version number
  @<file>      - read options from <file>
  --target=BFDNAME - specify the target object format as BFDNAME
 optional:
  --plugin <p> - load the specified plugin
 emulation options: 
  No emulation specific options
/usr/bin/ar: supported targets: elf64-x86-64 elf32-i386 elf32-iamcu elf32-x86-64 a.out-i386-linux pei-i386 pei-x86-64 elf64-l1om elf64-k1om elf64-little elf64-big elf32-little elf32-big pe-x86-64 pe-bigobj-x86-64 pe-i386 plugin srec symbolsrec verilog tekhex binary ihex
Makefile:102: recipe for target 'libapr-1.la' failed
make[4]: *** [libapr-1.la] Error 1
/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Build/Apr1StaticLib/build/apr_rules.mk:118: recipe for target 'all-recursive' failed
make[3]: *** [all-recursive] Error 1
external/CMakeFiles/Apr1StaticLib.dir/build.make:114: recipe for target 'ThirdParty/Stamp/Apr1StaticLib/Apr1StaticLib-build' failed
make[2]: *** [ThirdParty/Stamp/Apr1StaticLib/Apr1StaticLib-build] Error 2
CMakeFiles/Makefile2:171: recipe for target 'external/CMakeFiles/Apr1StaticLib.dir/all' failed
make[1]: *** [external/CMakeFiles/Apr1StaticLib.dir/all] Error 2
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2

Also @thanh-binh.to take a look through the forums to see if any other Ubuntu 16 topics help you. You might be fine just installing from binary as stated in another topic:

@thanh-binh.to We are still waiting to see the full cmake console output and the nupic.core SHA you are using. You can get the SHA by going into your git directory and running git rev-parse HEAD. Paste the output here. Also, when you provide your cmake output, please don’t paste the whole thing into this thread. Use a service like https://gist.github.com or http://pastebin.com.

I do not think so, because I did this additional step before in the version 14.04.
But I tried to do every step again (inclusive the step mentioned here), and have the same problem before.
I am 100% sure it is a problem with “invalid option -R of binutils”, but I really do not know where “/usr/bin/ar -R” is generated and called.

Best thanks.

@rhyolight here is the output of SHA:

git rev-parse HEAD
93dde655bcc7840d9dfffb616975294c95dc9ac2

The output of full cmake console:

binh@volkswagen:~/nupic/nupic.core/build/scripts$ cmake $NUPIC_CORE -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../release -DPY_EXTENSIONS_DIR=$NUPIC_CORE/bindings/py/nupic/bindings
Doing a local build
-- CMAKE BITNESS=64
-- CMAKE MEMORY=66805
GNU gold (GNU Binutils for Ubuntu 2.26.1) 1.11
Copyright (C) 2015 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.
Using ld.gold as LINKER.
-- INTERNAL_CXX_FLAGS_OPTIMIZED=  -fvisibility=hidden  -shared-libgcc -fdiagnostics-show-option -m64 -fPIC   -std=c++11 -fvisibility-inlines-hidden  -Werror -Wextra -Wreturn-type -Wunused -Wno-unused-variable -Wno-unused-parameter -Wno-missing-field-initializers -Wall -pipe  -O2 -mtune=generic -fuse-ld=gold -fuse-linker-plugin -flto-report -flto
-- INTERNAL_LINKER_FLAGS_OPTIMIZED=  -m64  -shared-libgcc  -Wl,--no-undefined -O2  -flto
-- EXTERNAL_C_FLAGS_UNOPTIMIZED=  -fvisibility=hidden  -shared-libgcc -fdiagnostics-show-option -m64 -fPIC  -Wno-unused-variable -Wno-unused-parameter -Wno-incompatible-pointer-types -Wno-deprecated-declarations
-- EXTERNAL_C_FLAGS_OPTIMIZED=  -fvisibility=hidden  -shared-libgcc -fdiagnostics-show-option -m64 -fPIC  -Wno-unused-variable -Wno-unused-parameter -Wno-incompatible-pointer-types -Wno-deprecated-declarations -pipe  -O2 -mtune=generic -fuse-ld=gold -fuse-linker-plugin -flto-report -flto
-- PYEXT_LINKER_FLAGS_OPTIMIZED=  -m64  -shared-libgcc  -O2  -flto -Wl,--allow-shlib-undefined
-- EXTERNAL_CXX_FLAGS_UNOPTIMIZED=  -fvisibility=hidden  -shared-libgcc -fdiagnostics-show-option -m64 -fPIC  -Wno-unused-variable -Wno-unused-parameter -Wno-incompatible-pointer-types -Wno-deprecated-declarations   -std=c++11 -fvisibility-inlines-hidden
-- EXTERNAL_CXX_FLAGS_OPTIMIZED=  -fvisibility=hidden  -shared-libgcc -fdiagnostics-show-option -m64 -fPIC  -Wno-unused-variable -Wno-unused-parameter -Wno-incompatible-pointer-types -Wno-deprecated-declarations   -std=c++11 -fvisibility-inlines-hidden -pipe  -O2 -mtune=generic -fuse-ld=gold -fuse-linker-plugin -flto-report -flto
-- EXTERNAL_LINKER_FLAGS_UNOPTIMIZED=  -m64  -shared-libgcc  -Wl,--no-undefined
-- EXTERNAL_LINKER_FLAGS_OPTIMIZED=  -m64  -shared-libgcc  -Wl,--no-undefined -O2  -flto
-- COMMON_COMPILER_DEFINITIONS=-DHAVE_UNISTD_H
-- COMMON_COMPILER_DEFINITIONS_STR= -DHAVE_UNISTD_H
-- EXTERNAL_STATICLIB_CMAKE_DEFINITIONS_OPTIMIZED=-DCMAKE_AR:PATH=gcc-ar;-DCMAKE_RANLIB:PATH=gcc-ranlib
-- EXTERNAL_STATICLIB_CONFIGURE_DEFINITIONS_OPTIMIZED_STR=AR=gcc-ar RANLIB=gcc-ranlib
-- CapnProto CXX_FLAGS=  -fvisibility=hidden  -shared-libgcc -fdiagnostics-show-option -m64 -fPIC  -Wno-unused-variable -Wno-unused-parameter -Wno-incompatible-pointer-types -Wno-deprecated-declarations   -std=c++11 -fvisibility-inlines-hidden  -DHAVE_UNISTD_H
-- CMAKE Found python interpreter /usr/bin/python2.7 version=%{PYTHON_VERSION_STRING}
-- PYTHON_EXECUTABLE   = /usr/bin/python2.7
-- PYTHON_INCLUDE_DIRS = /usr/include/python2.7
-- PYTHON_LIBRARIES    = /usr/lib/x86_64-linux-gnu/libpython2.7.so
-- CAPNP_EXECUTABLE          = /home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/bin/capnp
-- CAPNP_CMAKE_DEFINITIONS   = -DCAPNP_LITE=0
-- CAPNP_INCLUDE_DIRS        = /home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/include
-- CAPNP_LINK_LIBRARIES      = /home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/lib/libcapnpc.a;/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/lib/libcapnp.a;/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/lib/libkj.a
-- src_pycapnp_include_dir      = /home/binh/.local/lib/python2.7/site-packages
-- src_compile_flags =   -fvisibility=hidden  -shared-libgcc -fdiagnostics-show-option -m64 -fPIC   -std=c++11 -fvisibility-inlines-hidden  -Werror -Wextra -Wreturn-type -Wunused -Wno-unused-variable -Wno-unused-parameter -Wno-missing-field-initializers -Wall -pipe  -O2 -mtune=generic -fuse-ld=gold -fuse-linker-plugin -flto-report -flto
-- src_lib_static_nupiccore_compile_flags =   -fvisibility=hidden  -shared-libgcc -fdiagnostics-show-option -m64 -fPIC   -std=c++11 -fvisibility-inlines-hidden  -Werror -Wextra -Wreturn-type -Wunused -Wno-unused-variable -Wno-unused-parameter -Wno-missing-field-initializers -Wall -pipe  -O2 -mtune=generic -fuse-ld=gold -fuse-linker-plugin -flto-report -flto -I/usr/lib/python2.7/dist-packages/numpy/core/include
-- src_lib_static_nupiccore_link_flags =   -m64  -shared-libgcc  -Wl,--no-undefined -O2  -flto -static
-- src_lib_static_nupiccore_solo_link_libs = /home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/YamlCppStaticLib/lib/libyaml-cpp.a;/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Build/YamlStaticLib/libyaml.a;/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/Apr1StaticLib/lib/libapr-1.a;/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/AprUtil1StaticLib/lib/libaprutil-1.a;/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/ZStaticLib/lib/libz.a;/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/lib/libcapnpc.a;/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/lib/libcapnp.a;/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/lib/libkj.a;pthread;dl
-- PYEXT_LINKER_FLAGS_OPTIMIZED =   -m64  -shared-libgcc  -O2  -flto -Wl,--allow-shlib-undefined
-- CMAKE_CXX_COMPILER_ID        = GNU
-- src_swig_flags = -c++;-features;autodoc=0,directors=0;-noproxyimport;-keyword;-modern;-modernargs;-noproxydel;-fvirtual;-fastunpack;-nofastproxy;-fastquery;-outputtuple;-castmode;-nosafecstrings;-w402;-w503;-w511;-w302;-w362;-w312;-w389;-DSWIG_PYTHON_LEGACY_BOOL;-I/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/share/swig/3.0.2/python;-I/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/share/swig/3.0.2;-DHAVE_UNISTD_H;-DNTA_OS_LINUX;-DNTA_ARCH_64;-DHAVE_CONFIG_H;-DNTA_INTERNAL;-DBOOST_MATH_NO_LONG_DOUBLE_MATH_FUNCTIONS;-DBOOST_NO_WREGEX;-DNUPIC2;-DAPR_DECLARE_STATIC;-DAPU_DECLARE_STATIC;-DNTA_ASM;-DNTA_COMPILER_GNU
-- src_swig_link_libraries = /home/binh/nupic/nupic.core/build/scripts/src/libnupic_core.a;pthread;dl;/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/lib/libcapnpc.a;/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/lib/libcapnp.a;/home/binh/nupic/nupic.core/build/scripts/ThirdParty/Install/lib/libkj.a
-- Configuring done
-- Generating done
-- Build files have been written to: /home/binh/nupic/nupic.core/build/scripts

Do you have to have the latest codebase? If not, you could try the latest release binaries. They should work in Ubuntu 16. First, run these two commands a few times until they say there are no installations on your system:

pip uninstall nupic -y
pip uninstall nupic.bindings -y

Then run these two commands to install the binaries.

pip install https://s3-us-west-2.amazonaws.com/artifacts.numenta.org/numenta/nupic.core/releases/nupic.bindings/nupic.bindings-0.4.4-cp27-none-linux_x86_64.whl
pip install nupic

@rhyolight. Yes I have the latest codebase. I uninstall and install as you suggested, but hat a new error:
pip install nupic
Collecting nupic
Using cached nupic-0.5.4-py2-none-any.whl
Collecting pytest==2.5.1 (from nupic)
Collecting pytest-cov==1.6 (from nupic)
Collecting nupic.bindings==0.4.4 (from nupic)
Could not find a version that satisfies the requirement nupic.bindings==0.4.4 (from nupic) (from versions: )
No matching distribution found for nupic.bindings==0.4.4 (from nupic)

No, I meant do you need to have the latest codebase or will the latest release work?

Also, it doesn’t look like you ran this command? It is the command that installs nupic.bindings, which is the error you are getting.

pip install https://s3-us-west-2.amazonaws.com/artifacts.numenta.org/numenta/nupic.core/releases/nupic.bindings/nupic.bindings-0.4.4-cp27-none-linux_x86_64.whl

yes. I did before.
Here is the output of this command:

pip install https://s3-us-west-2.amazonaws.com/artifacts.numenta.org/numenta/nupic.core/releases/nupic.bindings/nupic.bindings-0.4.4-cp27-none-linux_x86_64.whl
Collecting nupic.bindings==0.4.4 from https://s3-us-west-2.amazonaws.com/artifacts.numenta.org/numenta/nupic.core/releases/nupic.bindings/nupic.bindings-0.4.4-cp27-none-linux_x86_64.whl
  Using cached https://s3-us-west-2.amazonaws.com/artifacts.numenta.org/numenta/nupic.core/releases/nupic.bindings/nupic.bindings-0.4.4-cp27-none-linux_x86_64.whl
Collecting pytest-cov==1.6 (from nupic.bindings==0.4.4)
Collecting numpy==1.9.2 (from nupic.bindings==0.4.4)
  Using cached numpy-1.9.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pytest-xdist==1.8 (from nupic.bindings==0.4.4)
Collecting pytest==2.5.1 (from nupic.bindings==0.4.4)
Collecting cov-core>=1.6 (from pytest-cov==1.6->nupic.bindings==0.4.4)
Collecting execnet>=1.0.8 (from pytest-xdist==1.8->nupic.bindings==0.4.4)
  Using cached execnet-1.4.1-py2.py3-none-any.whl
Collecting py>=1.4.19 (from pytest==2.5.1->nupic.bindings==0.4.4)
  Using cached py-1.4.31-py2.py3-none-any.whl
Collecting coverage>=3.6 (from cov-core>=1.6->pytest-cov==1.6->nupic.bindings==0.4.4)
Collecting apipkg>=1.4 (from execnet>=1.0.8->pytest-xdist==1.8->nupic.bindings==0.4.4)
  Using cached apipkg-1.4-py2.py3-none-any.whl
Installing collected packages: coverage, cov-core, py, pytest, pytest-cov, numpy, apipkg, execnet, pytest-xdist, nupic.bindings
Successfully installed apipkg-1.4 cov-core-1.15.0 coverage-4.2 execnet-1.4.1 numpy-1.11.0 nupic.bindings-0.4.5.dev0 py-1.4.31 pytest-2.5.1 pytest-cov-1.6 pytest-xdist-1.8

Something strange here. It looks like you still have a newer development version of nupic.bindings installed. I’m not convinced you have a clean slate. Again, run the following commands several times:

pip uninstall nupic -y
pip uninstall nupic.bindings -y

Now type pip freeze | grep nupic and paste the output, please.

@rhyolight: Here are the outputs using your commands

pip uninstall nupic -y
Cannot uninstall requirement nupic, not installed
pip uninstall nupic.bindings -y
Can't uninstall 'nupic.bindings'. No files were found to uninstall.
pip freeze | grep nupic
You are using pip version 8.1.1, however version 8.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
-e git+https://github.com/numenta/nupic.research@5230a39e3adf9f62aa16d146ad8415d0bfb9f7da#egg=htmresearch
-e git+https://github.com/numenta/nupic.core.git@93dde655bcc7840d9dfffb616975294c95dc9ac2#egg=nupic.bindings&subdirectory=bindings/py

These two eggs probably need to be removed. I can’t tell where they are coming from, though. I suggest you figure out how to uninstall those eggs before proceeding.