Nupic.Core C# Bindings

@chhenning

I found your fork the other day. Getting anything into VS 2017 would be a step forward. Definitely interested.

I tried using the CMake VS 2015 generator, since that seems to have been working and I’m getting the same issue - getting Cap’n Proto setup in Windows. Were you able to install pycapnp using pip?

Since python is the only language bound at the moment, I’m having difficult pinpointing how another language would be supported in core. So, to start, I guess I’d like to figure out how the python binding is working. Trouble is getting it built at the moment.

@heilerm Shoot me any questions or suggestions so it easier for others to work on the Visual Studio fork.

As for python, there are swig files in nupic.core\external\common\share\swig\3.0.2

Sorry, I meant: src\nupic\bindings . Look for the *.i files.

Just trying to reconcile the goal of the extraction plan with what’s in source, specifically a tutorial for creating language bindings.

I also tried to build nupic.core under VS 2017 and was unsuccessful. All I wanted was the ability to call into nupic core from C++ under Windows. For my project I did not need Python. From what I gather it is possible to build under Windows but only if I build under a very old version of VS and that is not an option in my case.

The algorithm modules in nupic.core such as TemporalMemory.hpp contain things like
#include <nupic/proto/TemporalMemoryProto.capnp.h>
I think this is generated by pycapnp. Since I cannot run or build pycapnp I was stopped.
This program is a wrapper around Cap’n Proto’s functions but I don’t know what it is intended to add. (http://jparyani.github.io/pycapnp/)

Cap’n Proto and all of the other third party modules have newer versions that do support Python 3 but pycapnp does not. Looking into pycapnp it appears that it is no longer being maintained. The last change was 7/2015.
So we would either have to update pycapnp ourselves (probably not likely) or change the nupic.core build to not use pycapnp. I have not investigated what it would take to do that. I suspect that if we were to get rid of pycapnp it would remove the restriction to using VS 2017 and also allow Python 3 because we could then link with the right libraries. Once nupic.core builds under VS 2017 we could then setup an interface for C# and any other languages people might want.

Building under VS 2017 will prevent Python2 from running with these newer libraries so we would have to upgrade everything to Python 3. A cursory check of all of the third party imports like Numpy I found they have Python 3 versions so that would not be a problem but more than likely we would break some existing Python code in Nupic in the process. There would be a lot of testing required to identify what was broken and fix them but it would be doable.

Tasks:

  1. Setup Community Fork
  2. Come up with a strategy for removing pycapnp
  3. Make changes to Nupic.core modules as needed.
  4. Make changes to build
  5. Run tests on all Nupic.core to confirm nothing is broken in the C++ code.
  6. Run tests on all existing Python code and fix any problems.

The new build must work for both Linux and Windows (under VS) and must also be able to be built with the old compiler such that it can still support Python 2 for those people that don’t’ want to upgrade quite yet.

I am a retired Software Engineer. I have some time and I would be willing to help in this task but I do not want to take on the project myself. Is anyone willing to help me?

Hi David,

It seems the desire to move to python 3 is overwhelming. I’m working on getting @chhenning branch to build so that I have something to look at. He’s upgrading to Python 3. I’m not an expert with C++, Python, CMake, VCPKG… any of this, really. That said, I’m very much willing to help… and learn. I’m not really sure why pycapnp is there. Looks like there’s some interop? C++ calling some numpy code? I’m on day 2 of absorbing all this.

If you’re going to do this in a community fork, why worry about Python 2 at all? Those not wanting to upgrade can simply continue using NuPIC stable releases.

@David_Keeney It looks like pycapnp supports Python 3 and just released a new version even 3 days ago.

@rhyolight mentioned seeing about getting official language bindings into master. Major changes would probably make that less likely. It is in maintenance mode… I’ll hope for a little and expect not a lot.

If noninvasive changes could be made in order to add the C# bindings, that could possibly go into master and then we could have community branches for the larger and riskier alterations like VS 2017 updates. Other binding could be added as well. I’m just not really sure how that would work. I can’t find a tutorial on how to do that, as outlined in the extraction plan. Any hope for that, @rhyolight? My thought was that once I got it to build using CMake, bindings would be added to ~\nupic\bindings\cs using swig and that’d be used to generate a C# project repository.

My immediate issue is installing pycapnp on Windows. I get the error described in my initial post. I suspected setuptools, but updating them didn’t work. Once I get that installed, I’m hoping that puts me past the CMake error I’m getting that’s related to it.

@heilerm Now that we have a new version of pycapnp it should make our work much simpler.

However, I still question why we are using pycapnp at all. I discovered that the data portions of the following classes are not defined in C++. They are declared in the nupic Python code. pycapnp builds the corresponding .h files so the C++ modules will compile.

ArrayProto
BitHistory
Cell
Cells4
ClaClassifier
ConnectionsProto
LinkProto
Map
NetworkProto
PyRegionProto
RandomProto.
RegionProto.
Segment
SegmentUpdate
SparseBinaryMatrixProto.
SparseMatrixProto
SpatialPoolerProto
SdrClassifier
SvmProto
TemporalMemoryProto
TestNodeProto
VectorFileSensorProto

This means that there is a compile time dependency on the Python nupic modules even if you are trying to build a C# or even a C++ implementation. I suspect this is so that these objects can be passed by value or the object’s public fields can be directly accessed by the Python modules. This requires serialization and deserialization of the object on each access. Not a very good design for a core library but this is probably not the time to be re-designing the library.

Lets get a new CMakelist.txt working for nupic.core using the new cycapnp and then we can decide what to do.

A tutorial on creating a language binding in swig? No we don’t have one. But I’m sure swig has docs. You are pioneering new territory here folks. As you make progress, please track it here on #nupic:developers. This forum is also a wiki and a great search tool. It would be great to have a wiki page on creating language bindings.

Hmm. I think I need a visual of what’s going on. A map. Working on a diagram. At work, so I might not have anything until later.

@David_Keeney Do you think a test/prototype project would be a good idea? C+±only. Copy/paste a couple classes from nupic.core. Pick two languages for binding. Keep the same/similar project structure. Test it without having to support legacy, on Windows, Linux, and OS X? No python anything. The core extraction plan describes it’s goal to be C++ only, so I too was surprised to find a dependency on python. That plan describes:

nupic.core - C++

nupic-<flavor> - language binding
nupic-<flavor>-client - language client

As an alternative, maybe:

nupic.core

nupic.bindings - shared binding module? necessary?
nupic.bindings.<flavor>

nupic.<flavor>

Then we can have that to discuss and maybe see if we can do anything for master?

@rhyolight Maybe a new topic, once we get things ironed out? See what we need to do. Not sure this could be merged into master, due to its current state and the impact of this work.

I could be way off, but this is what I’m thinking, as I understand it:

SETUP -> BUILD -> PUBLISH

SETUP

This is kind of a black box right now, so correct me if I’m wrong.

  1. With the source code (*.hpp, *.cpp), generate swig interfaces (*.i), then generate whatever wrapper(s) (*.cs, *.py, etc.) you want? Can’t this be scripted?

  2. Using the wrappers and CMake, generate a build system?

BUILD

System-specific. Whatever is available in CMake generator tool?

PUBLISH

System-specific. Package DLLs and PY and whatnot.

PROTOTYPE

To start, I’d propose…

nupic.core (prototype)
	/env
	    // cmake generator output?
	/gen
	    // generated swig input/bindings (*.i)		
	/obj
	    // generated swig output/wrappers (*.cs, *.py., etc.)
	/src

        // flatten this out a little bit from src/nupic? put test and examples in other projects?

		/algorithms
		/bindings
		    // can't this be generated from headers using swig? whats the history behind this? am i misunderstanding swig?
		/encoders
		/engine
		/math
		/ntypes
		/os
		/proto
		/py_support
		    // is this why nupic.core depends on python? move that to a different repo and pull it in - isolate it?
		    // would that simplify the build process? are things just fine?
		    // python is still required to run C++? is numenta really cool with that? to run... java, you would need C++, java, and python runtimes, no?
		/regions
		/types
		/utils

    // setup can go in the root
    // setup script does:
    // 1. generate swig interfaces from headers into gen
    // 2. generate wrappers from swig interfaces into obj

Actually, no build process would be necessary for the wrappers.
Just:

GENERATE -> PACKAGE?

Setup CMake for the C++ project.

No… back to the drawing board.

Maybe this is a better attempt…

SOURCE (C++)

  1. CMAKE (Generate a build system with ~/src)
    CMake the build system for your environment.
\*.cppproj
\*.sln
...
  1. BUILD
    Use that build system.
nupic.core.lib
nupic.core.dll
...

GENERATED (e.g., C#, Python)

  1. WRAP
    Use Swig? Can those interfaces be generated from the headers, then used to wrap the source code?
    Script this? (e.g., wrap.py -python, wrap.py -csharp).
  • generate swig interfaces from headers into ~/obj.
  • generate code from swig interfaces into ~/gen.
  1. CMAKE (Generate a build system with ~/gen)
    CMake the build system for your environment.
\*.csproj
\*.sln
...
  1. BUILD
    Use that build system.
// C#
	nupic.core.cs.dll
// PYTHON
	nupic.core.py.py

PUBLISH

Build the source and generated code.
Package it and ship it.
NPM? Nuget? Other(s)? All-the-above? Up the community to provide support?

// C++
	// pass-through?
	nupic.core.lib		// C++
// C#
	nupic.core.dll		// C++
	nupic.core.cs.dll	// C#
// PYTHON
	nupic.core.lib		// C++
	nupic.core.py.py 	// Python

PROTOTYPE

To start, I’d propose…

nupic.core (prototype)
	/gen
		// generated swig output\wrappers (*.cs, *.py., etc.)
		// gitignored
	/obj
		// generated swig input\bindings (*.i) generated from source (*.hpp, *.cpp)
		// gitignored
	/src
		/algorithms
		/bindings
			// can't this be generated from headers using swig? whats the history behind this? am i misunderstanding swig?
		/encoders
		/engine
		/math
		/ntypes
		/os
		/proto
		/py_support
			// we can talk about removing this at a later point
			// since this is exposed, i wouldnt touch it - thats high-impact
			// ignore this until the end of the prototype?
		/regions
		/types
		/utils

	// project.cmake - setup build system for source code (C++)
	// project-cs.cmake - setup build system for generated code (C#)
	// project-py.cmake - setup build system for generated code (Python)

	// publish.py - publish script
	// publish-cs.py - publish script wrapper (C#)
	// publish-py.py - publish script wrapper (Python)

	// wrap.py - python script? have to pick something for both linux and windows...
	// wrap-cs.py - ease-of-use?
	// wrap-py.py - ease-of-use?

@heilerm That looks like the right idea. We probably should have a /build folder. This thing needs a ton of third party stuff and we need a place to put all of it.

nupic.core is distributed with patches for those third parties, too. Got a little bit farther with the Windows build and currently have an issue getting those to patch, but… there they are.

I have managed to get the CMake to load up in Visual Studio 2017 and it tries to compile but some of the third party packages are breaking. For example the Yaml-cpp sub cmake is creating a library that has a slightly different name than expected and it breaks the build. About half of the 30 or so packages are breaking so there is some work to do.

Here is what I have so far as a procedure to build under Visual Studio 2017.

  1. Install the prerequisites:
    Visual Studio 2017 from https://www.visualstudio.com/downloads/ (Community version is free)
    This has the Visual C++ 14.0 compiler.

    CMake install from https://cmake.org/

    GIT install from https://git-scm.com/downloads
    Includes patch.exe required when building external libraries using Visual Studio.
    Make sure C:\Program Files\Git\usr\bin is added to your PATH environment variable
    and restart your Command Prompt window if you have one open.

    Python Download and install Python 3.6.x (64bit version) from Python.org.
    Be sure to click the checkbox to have it set the environment variables.
    If you already have Python 2.7.x installed it is ok. This will place
    the python path environment variable for Python 3.6 ahead of those for
    Python 2.7 so that typing “python” will get Python 3.6.

        Opened a Command Prompt window and verified that it has the right python version.
    
            C:> python
            Python 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)]
             on win32
            Type "help", "copyright", "credits" or "license" for more information.
            >>> 
        Type ctl-Z CR to exit python.
    

    pip This is installed with Python but you need to update it to make sure it is the latest.
    C:> python -m pip install -U pip

    Python support modules
    Start Command Prompt "as Administrator"
    C:> pip install -U cython
    C:> pip install -U setuptools
    C:> pip install wheel
    C:> pip install --user numpy
    It should get the latest version of numpy (numpy-1.13.3-cp36-none-win_amd64.whl)

  2. Clone nupic.core repository
    C:> cd \xxx (or where you want the repository to be)
    C:\xxx> git clone https://github.com/numenta/nupic.core.git

  3. Use CMake to create the Visual Studio solution and project files
    C:\xxx> cd nupic.core
    C:\xxx\nupic.core> set NUPIC_CORE=%CD%
    C:\xxx\nupic.core> mkdir build\scripts
    C:\xxx\nupic.core> cd build\scripts

    Run cmake to generate MSVC 2017 solution and project files
    C:\xxx\nupic.core\build\scripts> cmake -G “Visual Studio 15 2017 Win64”
    -DCMAKE_INSTALL_PREFIX=…\release
    -DPY_EXTENSIONS_DIR=…\bindings\py\src\nupic\bindings

  4. Start up Visual Studio
    C:\xxx\nupic.core\build\scripts> nupic_core_main.sln
    Make sure to select the Release x64 configuration then build the ALL_BUILD project.

Once we get this running there are some install steps to get things in the right places. Then we have to create the bindings for Python, C#, and whatever else.

I think I’m at that point. Crash course in C++, Python, CMake, and building in multiple environments.
Had to install GnuWin32 for patch.exe.

7 succeeded, and then starts to fail at yaml-cpp-bundle.

Going to start the prototype. I’ll share it when I get it up. I just need to learn all this bit by bit without all of nupic.

@heilerm I made some progress today. I still have 12 out of 29 projects that are failing. Almost all of them have to do with the Python interface.

Again, what I am trying to do is get the existing Nupic.core codebase compiling on Visual Studio 2017 with Python 3. Once that is working we can look at ways to extract the Python bindings into a separate build and then add the C# bindings in the same way.

So here are some changes you can make. Like I said, this does not all work yet.

+++++++++++++++++
When building under Visual Studio, the yaml-cpp library is built with a slightly different name.
Generated file is F:\AI\community\nupic.core\build\scripts\ThirdParty\Build\YamlCppStaticLib\Release\libyaml-cppmd.lib

nupic.core/extrnal/YamlCppLib.cmake line #44
Change

# Path to static yaml-cpp installed by external project
set(yamlcpplib_built_archive_file
    "${yamlcpplib_install_lib_dir}/${STATIC_PRE}yaml-cpp${STATIC_SUF}")

to

# Path to static yaml-cpp installed by external project
if(MSVC)
    set(yamlcpplib_built_archive_file
        "${yamlcpplib_install_lib_dir}/lib${STATIC_PRE}yaml-cppmd${STATIC_SUF}")
else()
    set(yamlcpplib_built_archive_file
        "${yamlcpplib_install_lib_dir}/${STATIC_PRE}yaml-cpp${STATIC_SUF}")
endif()

+++++++++++++++++
7> LINK : fatal error LNK1181: cannot open input file 'F:/AI/community/nupic.core/build/scripts/ThirdParty/Build/YamlStaticLib/yaml.lib’
Generated library file is at ‘F:/AI/community/nupic.core/build/scripts/ThirdParty/Build/YamlStaticLib/Release/yaml.lib’

nupic.core/external/YamlLib.cmake line #38
change

# NOTE Yaml lib doesn't have an install target and leaves artifacts in build dir
set(yamllib_build_dir "${EP_BASE}/Build/YamlStaticLib")

to

# NOTE Yaml lib doesn't have an install target and leaves artifacts in build dir
if(MSVC)
    set(yamllib_build_dir "${EP_BASE}/Build/YamlStaticLib/Release")
else()
    set(yamllib_build_dir "${EP_BASE}/Build/YamlStaticLib")
endif()

+++++++++++++++++
9> c1xx : fatal error C1083: Cannot open source file: ‘Files/Python36/Lib/site-packages/numpy/core/include’: No such file or directory
This path should be C:/Program Files/Python36/Lib/site-packages/numpy/core/include
There is a problem with the space in filename: “Program Files”.

nupic.core/src/CMakeLists.txt Line#189
change

#
# Locate Python artifacts
#
find_package(PythonLibs REQUIRED)
include_directories(SYSTEM ${PYTHON_INCLUDE_DIRS})

to

#
# Locate Python artifacts
#
find_package(PythonLibs ${PYTHON_VERSION_STRING} REQUIRED)
include_directories(SYSTEM ${PYTHON_INCLUDE_DIRS})

nupic.core/src/CMakeLists.txt Line#129
change
#

# Find Python Interp, min. v2.7
# And NumPy core directory
#
find_package(PythonInterp 2.7 REQUIRED)
message(STATUS "CMAKE Found python interpreter ${PYTHON_EXECUTABLE} version=${PYTHON_VERSION_STRING}")

to

#
# Find Python Interp, min. v3.x
# And NumPy core directory
#
# There was a bug in CMake before Version 3.0 where find_package returned the wrong python parameters.
cmake_minimum_required(VERSION 3.0)
find_package(PythonInterp 3 REQUIRED)
message(STATUS "CMAKE Found python interpreter ${PYTHON_EXECUTABLE} version=${PYTHON_VERSION_STRING}")