Proposal to introduce pybind for move toward Python 3 compatibility


#81

Yeah, that can wait. They’re included for now. Maybe we can include that in the scope of general CMake changes? We can do a lot to reduce the complexity of the CMake scripts. I have a few question on the matter, but that should be a different thread.


#82

Excellent! I’ll incorporate it into my repo first before pushing my changes.

For a user it’s easy. For cmake it should be too.

OK. I’ll start rebasing again in a few hours. Will keep you updated.


#83

Added a topic for CMake Enhacements and Package Management.


#84

The first step in rebasing is done. Have a look at step_1

Things I have done:

  1. Emptied out /external/

  2. Removed /bindings/

  3. Added /include

  4. Added boost 1.66 into /external/boost

  5. Added boost headers to /include/boost

  6. Added zlib 1.2.11 into /external/zlib

  7. Added yaml-cpp 0.5.3 into /external/yaml-cpp

  8. Added yaml-cpp headers into /include/yaml-cpp

  9. Added pybind11 into /external/pybind11

  10. Added pybind11 headers into /include/pybind11

  11. Added gtest 1.8 to /external/gtest

  12. Added gtest headers to /include/gtest

Let me know you think.


#85

Will make a PR and discuss there, thanks!!
Also, I set up a ‘nupic.cpp’ team on GH, you have an invitation pending (others please PM me your GH username if you want to join the reviewers)


#86

@breznak Should we create a new repo and then override the old nupic.core when we are done? Just thinking…


#87

@breznak I have added back the bindings/ folder. Let me make sure the tests are running through. I have checked in a while. Probably be done tomorrow.


#88

@chhenning what is your feeling about our tries to integrate the pybind in the repo so far?
I think we got diverged and are now solving issues of {repo structure, platform support, dependency management}*.

Our target here was to introduce pybind, ideally replace SWIG, make Regions use pybind functionality.
Should we try to do that with minimal changes?

  • add pybind headers where other headers are now
  • replace PyRegion with your new implementation
  • ensure all tests pass (do you think it’s possible? Even considering nupic (py) depends on it?)
  • drop SWIG and other libs

…Next, go to Py3

*) which are important and will be addressed. but separately.


#89

@breznak Here is my thinking of what we should do.

1. Incorporate my nupic.core code changes. Ideally copy over /src/nupic and /src/test with my code.

  • You can take src from here
  • You can do that on separate branch or a new repo.
  • Add pybind11 to /include
  • It doesn’t matter what python you have on you system.

2. Compile nupic.core and only that. Don’t think about the python bindings modules.

  • You will have to adapt the cmake script. Some cpp files are deleted (think old python c api wrapper classes)
  • Ignore /src/nupic/python files. These are just for the bindings modules.
  • There might be some compiler/linker error we can discuss separately.

3. Compile and run unit tests.

All these steps don’t account for python. That’s next.

What do you think?


#90

Sounds like a feasible, minimalist approach to me. All clear, I’ll do that tonight.

Compile nupic.core and only that. Don’t think about the python bindings modules.

Will do that. Then unit-tests should pass, the py.test bindings/py would be N/A.

  • Is it on your next step, to bring the NAPI functionality back?

#91

The unit-tests should run and pass. There might be one fail but we can fix that later.

Don’t run “py.test bindinds/py” I’m still working on it.

What is “NAPI functionality”?


#92

ok :+1:
sure (I’ll keep it in a feature PR, merge once the tests pass, as bindings is essentially what pybind touches)
Network & Regions, basically what the tests above cover


#93

I have added python bindings tests. It’s all python 3 now. There are still some assertion errors but nothing too major.


#94

Nice! I saw @breznak closed the PR in anticipation of another. Is this what you two were waiting for in order to get it into the community fork?


#95

Not really. @breznak wanted to open a new pull request based on what I wrote:


#96

I just found this thread and checked out your git repo. Looks like you’ve made some great progress on porting to Python 3. The last update in the Git repo seems to be about 2 months ago. Is active development still ongoing?


#97

Yes, work is still continuing on the Community Fork but progress has been slow.


#98

Hi all, my hope is that I can return to the project next monts. Work has been crazy unfortunately…

BTW, the latest gcc and Visual Studio have native support for c++'17 filesystem. That would mean no boost::filesystem dependency!