Can't install htm.core on linux

Background

Hi guys, I’m a new HTM learner and need some help. I’m intended to install htm.core on Ubuntu 16.04.6 LTS (GNU/Linux 4.4.0-142-generic x86_64). And I tried install and failed. It shows that I have met all the prerequisites for installation:

  • anaconda python 3.7+
  • cmake version 3.7
  • g++-6 which is c++11 compatible

Feedback of install from pypi

I know I can use that simple command to install

python -m pip install -i https://test.pypi.org/simple/ htm.core

but it shows

ERROR: Could not find a version that satisfies the requirement pytest==4.6.5 (from htm.core) (from versions: 0.0.0.dev1)
ERROR: No matching distribution found for pytest==4.6.5 (from htm.core)

Feedback of building from source

CMake Error at src/ExpandStaticLib.cmake:68 (message):
  ExpandStaticLib: obj extraction process failed exe_result='1'
Call Stack (most recent call first):
  src/CMakeLists.txt:298 (ExpandStaticLib)


-- Configuring incomplete, errors occurred!
See also "/home/xxx/htm.core/build/scripts/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
  File "setup.py", line 374, in <module>
    getExtensionFiles(platform, build_type)
  File "setup.py", line 239, in getExtensionFiles
    generateExtensions(platform, build_type)
  File "setup.py", line 282, in generateExtensions
    configure(platform, build_type)
  File "setup.py", line 341, in configure
    subprocess.check_call(["cmake",BUILD_TYPE , PY_VER, REPO_DIR])
  File "/home/xxx/.conda/envs/htm/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '-DCMAKE_BUILD_TYPE=Release', '-DBINDING_BUILD=Python3', '/home/xxx/htm.core']' returned non-zero exit status 1.

Same issue has been raised on github


I’m really appreciated if anyone can fix my problem!

Thank you for using htm.core. You are almost there.
Use the standard Python from python.org rather than Anacondra Python.

Although some people have been able to get Anacondra to work, the library was built and tested using standard Python. Anacondra is not supported.

The pypi download will work only for the most recent version of Ubuntu and python. So you will need to build from sources.

https://test.pypi.org/simple/hexy/?

john@john-VirtualBox:~$ python --version
Python 3.7.5
john@john-VirtualBox:~$ python -m pip install -i https://test.pypi.org/simple/ htm.core
Collecting htm.core
  Downloading https://test-files.pythonhosted.org/packages/26/d0/adff7d1cf6650d4806cfabb5451ee3f93e575116eaca5121afd7998c97c3/htm.core-2.0.18-cp37-cp37m-manylinux1_x86_64.whl (2.6MB)
    100% |████████████████████████████████| 2.6MB 574kB/s 
Collecting hexy>=1.4.3 (from htm.core)
Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python3/dist-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 554, in _prepare_file
    require_hashes
  File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 423, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/usr/lib/python3/dist-packages/pip/index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 795, in get_page
    resp.raise_for_status()
  File "/usr/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://test.pypi.org/simple/hexy/
john@john-VirtualBox:~$ ll

I’ve tried installing htm.core a dozen times now, and reinstalled ubuntu VM on my windows laptop, and no luck. Any help would be greatly appreciated. Is there a recommended OS that I should install instead of ubuntu-18.04.4-desktop-amd6?

(1) Simple Python build (any platform)

** At a command prompt, cd to the root directory of this repository.**

** Run: python setup.py install --user --force

(2) Simple C++ build

After downloading the repository, do the following:

cd path-to-repository
mkdir -p build/scripts
cd build/scripts
cmake …/…
make -j install

They both crash at the same point due to thinking the folder …/cmake is a Boost library, here it is from build/ThirdParty/results.txt:

common_INCLUDE_DIRS@@@/home/john/htm.core/external/common
common_LIBRARIES@@@/home/john/htm.core/build/ThirdParty/libcommon.a
yaml_INCLUDE_DIRS@@@/home/john/htm.core/build/ThirdParty/libyaml/libyaml-src/include
yaml_LIBRARIES@@@/home/john/htm.core/build/ThirdParty/libyaml/libyaml-build/libyaml.a
yaml_DEFINE@@@YAML_PARSER_libYaml
eigen_INCLUDE_DIRS@@@/home/john/htm.core/build/ThirdParty/eigen/eigen-src
BOOST_ROOT@@@/home/john/htm.core/build/ThirdParty/boost/Boost_download-src
Boost_INCLUDE_DIRS@@@/home/john/htm.core/build/ThirdParty/boost/Boost_download-src
Boost_LIBRARIES@@@/home/john/htm.core/build/ThirdParty/boost/Boost_download-src/stage/lib/cmake@@@/home/john/htm.core/build/ThirdParty/boost/Boost_download-src/stage/lib/libboost_filesystem.a@@@/home/john/htm.core/build/ThirdParty/boost/Boost_download-src/stage/lib/libboost_system.a
gtest_INCLUDE_DIRS@@@/home/john/htm.core/build/ThirdParty/gtest/googletest-src/googletest/include
gtest_LIBRARIES@@@/home/john/htm.core/build/ThirdParty/gtest/googletest-build/googletest/libgtest$<$CONFIG:Debug:d>.a
mnist_INCLUDE_DIRS@@@/home/john/htm.core/build/ThirdParty/mnist_data/mnist-src/include
mnist_SOURCE_DIR@@@/home/john/htm.core/build/ThirdParty/mnist_data/mnist-src
cereal_INCLUDE_DIRS@@@/home/john/htm.core/build/ThirdParty/cereal/cereal-src/include
digestpp_INCLUDE_DIRS@@@/home/john/htm.core/build/ThirdParty/digestpp/digestpp-src
cpp-httplib_INCLUDE_DIRS@@@/home/john/htm.core/build/ThirdParty/cpp-httplib/cpp-httplib-src

This should have worked. I am looking into it.

I think our unit tests missed this because none of them build with Boost anymore. Regardless, we should make sure it will work for those that use an older compiler.

Boost is not required when using a compiler that can provide the filesystem module. In your case, you could use cpp-8 with ubuntu 18 and it will not require Boost.

In the mean time I will try to figure out what broke the build the uses Boost.

Sorry, That should have been the
'gcc-8' package, not ‘cpp-8’.

Actually I don’t care what compiler I use, I just want to get an HTM installation running where I can start exploring the code and adding some features for AI modeling I am trying to do. Since I am installing a VM with the latest linux os I can find, I would think it should have the latest compilers. Can you guys recommend a linux distribution that HTM would work on out of the box? I’ll try the gcc-8 advice.

The best to get htm to work in to use the minicoda distrubution for linux (x64). Yes you do get those messages during install , just go through and install then manually ( pip install ) , some of them are version specific so install the version for which you are getting the error. you might have to do this for all other dependencies , after this it works fine. I have gone through the same problem , it works fine . IMPORTANT use miniconda not full Anaconda. I have tested this on Ubuntu 18.04 LTS (x64)

I just installed htm.core on Ubuntu 19.10 which uses gcc-8 by default. This is a build from sources on a fresh clone using the standard Python 3.7.5 from python.org.

I am setting up Ubuntu 18 to confirm it still works.

1 Like

Thank you very much. I’m trying to fix that.

But before I successfully install the package, I would like to know If there is any API document that can help me learn htm.core?

I ask this because I found that the API settings were different from nupic. Nupic has its well written document while I can’t find any tutorial for htm.core, which make it very hard for new learner like me (stupid)…

If you are newby than go watch this end to end https://www.youtube.com/watch?v=XMB0ri4qgwc&list=PL3yXMgtrZmDqhsFQzwUC9V8MeeVOQ7eZ9

After this understand what are SP and TP in htm context and how to encode data for them. Finally see coding examples in HTM core github repo. You can check out my repo for HTM , all code is fully commented. Circle back if you have any question .

@aamir121a I am very interested in how do you compare the performance or quality of your HTM image encoder with autoencoders?

Well I do not have any specific number to show you, however there are a few problems to over come before you can effectively use Auto encoders as input to SP to TP, One, information is lost when using dimensinal reduction ( 128 to 3 ) + norm of original vector = 4. The reason dimensional reduction is required is the SP itself ( in current implementation ) cannot deal with 128 D directly, images encoding each element of 128 D ( float element ) to RDSE with 1000 bits, this would produce 128000 bit array as input to SP, with SP size itself being 128000.

While my expriment was the step in right direction and using this technique will allow us to encode NLP and vision as input to SP and TP , however I I feel the the developers, researcher at HTM core have figure out a way to effeciently scale core frame work to much more data than previously attempted.

FYI HTM core now has a new vision encoder (eye) I have not trid this yet.

thanks