How to deal with pycapnp / Capnp dependency in nupic.core

serialization

#1

Regarding the following issues:

https://github.com/numenta/nupic.core/issues/896

 

https://github.com/numenta/nupic.core/issues/701

I have had several experiences myself (and reports from users) where Cap’n Proto needed to be manually installed on the system before the nupic.core installation would work, even though it should be getting installed as a part of the pycapnp dependency.

Last I heard about these tickets, we were waiting on some movement on the Windows side, but I don’t know what the current state is.

The big question in my mind is: How should Cap’n Proto be installed?

Do we require users to manually install it? Do we try to figure out why it doesn’t always work with we just use pycapnp's dependency installation? Or do we try to force an installation within our own build config? Why does pycapnp insist on installing Capn even when I have it installed manually?


#2

not entirely on topic, but about “dealing with py/Capnp”:
I’ve had issues with it and/or I don’t need any serialization for many use-cases, I have already proposed encapsulating the serialization into an interface which could offer a dummy serialization option that would do nothing.


#3

See https://discourse.numenta.org/t/nupic-core-and-pycapnp-dependency-discussion/1246/3 for recent activity on this subject.


#4

There are several phases to consider:

  1. nupic.core build (cmake + make) relies on pycapnp being installed for its embedded capnproto headers, which nupic.core uses during compilation. Are we trying to solve this specific case?
  2. Subsequently, you can both generate and install the nupic.bindings wheel without having pycapnp installed. So, pycapnp is not an issues in this phase.
  3. Using nupic.bindings in python application or test:
    3.1 If you’re using nupic.bindings through nupic, then nupic as of this week has a conditional dependency on pycapnp. Therefore, installing nupic would install the correct version of pycapnp automatically. This scenario is no longer problematic (after the latest changes are released).
    3.2 However, you would need pycapnp if you want to test raw nupic.bindings directly without nupic. Are we also concerned about this specific scenario at this very moment?