Set up CI services

ci
tooling

#1

Can someone help setting up the CI?

We can discuss here what service to use… Or what other features we want from CI?

But my assumptions are:

  • want to cover all supported platforms: lin/win/osx
  • 64/32 bit
  • for both c++, py repos

I have opened Issues in both repos, and would love to merge PRs. I guess for starters the same services would be great, as we can just change the tokens?


#2

I need to get cmake going first on mine. But for Windows it’s Appveyor and for Linux/Mac it’s Travis or bamboo.


#3

@rhyolight would be the one to weight in, numenta tried all of them and currently ended up with

  • circleCI
  • appveyor
  • bamboo

Any insights if/why those are the best?


#4

Different platforms different CI. :wink:


#5

I honestly think you should keep it simple and start with just Travis-CI and AppVeyor, which will cover your major deployment platforms: OS X, Linux, & Windows. There are old travis-ci config files in the history of the repos you could reference. They contain configurations for both OSX and Linux.

We added bamboo with the intention of eventually doing everything in bamboo, which was a bit of a stretch goal that we haven’t accomplished. That being said, it comes in quite useful for releases and deployments. We’ve automated a lot of things in our pipeline with bamboo. If you think it would be useful, I think I could turn over some bamboo resources for community usage, although I can’t help set up services.


#6

Thanks for the memo. I’ll go with Travis and AppV for c++ repo first, then for Py repo.
We won’t have a build service running, so the tests using new bindings will fail.


#7

I have Windows CI running (with badge + releases) here

Is there a version with CI for Linux? Seems the support has been dropped from numenta/nupic.core and all is just for OSX?!


#8

We build nupic.core for Linux in Bamboo:

https://ci.numenta.com/browse/NUP-CORE-644


#9

But before we had a release pipeline in bamboo, we had a Travis-CI build for nupic.core. You should be able to find it if you go back in history about a year.


#10

Found an unexpected problem, TravisCI builds for OSX take ages to start, seems to be known problem

I have a working PR, but will have to move to CircleCI for that reason with OSX (need to shoot them an email to get the OSX for free for OSS projects…)

EDIT:
Logged in to CircleCI with my github name (breznak), but I cannot set it up for htm-community/nupic.cpp , as I’m not an “org admin”, as the Circle requests. @rhyolight will have to do that, please.


#11

You are now an owner of the htm-community github org. I think that is what they want, so give it another try.


#12

Good news, everyone.

  • CI works (Win/AppVeyor,64b,MinGW, Linux/Travis,64b,gcc4.8; OSX/Travis,clang)
    • OSX build is too slow (Travis’ fault, might need to move to CircleCI)
  • releases somewhat work (I generate the whl artefacts, but the GH releases contains “some folder snapshot”)
  • all unit tests passing OK
  • badges for green passing :wink:
  • pypi releases not addressed yet
  • managed a nice cleanup of the .yml files.

Thanks @rhyolight


#13

That’s great, @breznak!! A couple of the badges are still pointing to the old nupic builds though (travis & circleci). Let me know when you get all building and green for by cpp & py, then I’d like to announce this project officially on the #nupic forum (if you like).

I hope everyone else is one board here? @chhenning @heilerm ? I think this is a great direction forward for you guys. Marek knows nupic better when anyone outside Numenta, and he’s an experienced committer on the project.


#14

I’m aboard. :slight_smile:


#15

I’m totally lost.

@David_Keeney had a fork with upgrades to Python 3, which was good. So I was looking at working on that.

@chhenning then had a brand new repository with a lot of good stuff, which was good, but different - arguably a better, more modern approach - but not a fork. So I was looking at working on that.

Now we have official community forks, and their structures are different from the single repository @David_Keeney, @chhenning, and I agreed on. That’s fine, just another pivot. Now I’m looking at working on that. Is @David_Keeney merging his fork into that? What’s happening to @chhenning’s stuff? Are we using multiple repositories now, or is that still up in the air.

I’ll have to get caught up with the plan for those forks. I have no idea what to work on or where. It’s a little hard to track this plan on the forum in its current format. Wouldn’t the Github issue tracker and wiki be better to fold into, now that we have community forks? It’s hard to get into anything with this much volatility. Link to that/those in a pinned thread in the community fork forum?


#16

But I’m all for the CI plan.


#17

The plan is to move my stuff into “HTM Community” repo. This might take a few days. If you like to work on something I suggest you use my repo for now. I’ll make sure it wont get lost.

Does that help?


#18

Got it. Might ask a few questions in the correct topic.


#19

Basically I’d just like to ensure continuity for you and all other people who base their work on nupic.core forks.
The intention is not to get anyone’s (to a managable degree) work lost, that’s why I discussed with @chhenning to rebase (fork) his Py3 work on top nupic.core.
I’m trying to maintain the htm-community/nupic.cpp repo which has tests and functionality same (or better in the future) as the original Numenta’s.
Currently the tests and builds are available there. And I have some little fixes to code scattered there.
We’ll be working with Christian to merge his code drop into that repo in more understandable (smaller) pieces.
After that would come @David_Keeney 's work on bindings and C# (and others) clients.

TL;DR which fork currently to base your work on? If you depend on py3 stuff, use


If it’s not related, go with

which is basically nupic.core for now.

Actually, this bazaar is a great example of a situation I want to avoid: everyone jumps their fork but the work then becomes incompatible. So I suggest we’d try to merge all the new features iteratively into a common community repo. It’s also easy to add commiters and maintainers there to keep it running.


#20

:+1: :100: Really important for an open source project for everyone to get behind one effort.